2024-04-27T07:37:07.234 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T07:37:07.234 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} --owner scheduled_teuthology@teuthology --verbose --name teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096 -- /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/orig.config.yaml 2024-04-27T07:37:07.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T07:37:07.377 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672096' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Reduced data availability - Degraded data redundancy sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4710 sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f sleep_before_teardown: 0 subset: 356/512 suite: fs suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBK5//N9I5E8X52d9w0riAGxZo/yEWmWVfz5IcTd0tHDDmpIi0mfkrfZpCBvNn7ogMpSAenSCGWIcXBnCuYlgvc= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII9zDJkhPI0/3gDcjcctfIsRaLoXjYWnEsO4xBm3NFpKrd5ss4yUnjoi6O8bJpJ1kOqbgWwtLoG3PIc9Eesj4k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_20:24:13 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-27T07:37:07.377 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa; will attempt to use it 2024-04-27T07:37:07.378 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks 2024-04-27T07:37:07.378 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T07:37:07.383 INFO:teuthology.task.internal:Checking packages... 2024-04-27T07:37:07.407 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f' 2024-04-27T07:37:07.408 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T07:37:07.408 INFO:teuthology.packaging:ref: None 2024-04-27T07:37:07.408 INFO:teuthology.packaging:tag: None 2024-04-27T07:37:07.408 INFO:teuthology.packaging:branch: main 2024-04-27T07:37:07.408 INFO:teuthology.packaging:sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:37:07.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-04-27T07:37:07.629 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3253-g4e213a58-1jammy 2024-04-27T07:37:07.630 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T07:37:07.638 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T07:37:07.638 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T07:37:07.643 INFO:teuthology.task.internal:Saving configuration 2024-04-27T07:37:07.660 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T07:37:07.666 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T07:37:07.691 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 07:31:40.686952', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB51yrJP7G+AwZb0oaz22Sh0ZDW+Rae15I57hvLDx1eF'} 2024-04-27T07:37:07.713 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 07:31:40.688379', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKl/UN0I2zc2nC6e5+X6eZ6aFAupok7pbwAQwHK7GzAG'} 2024-04-27T07:37:07.714 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T07:37:07.720 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-27T07:37:07.720 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-27T07:37:07.720 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T07:37:07.794 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feaeb450ee0>, signals=[15]) 2024-04-27T07:37:07.795 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T07:37:07.801 INFO:teuthology.task.internal:Opening connections... 2024-04-27T07:37:07.801 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:37:07.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:37:07.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:37:07.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:37:07.961 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T07:37:07.968 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-04-27T07:37:07.974 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-04-27T07:37:07.974 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="22.04" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=jammy 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-27T07:37:08.022 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-27T07:37:08.023 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-27T07:37:08.023 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=jammy 2024-04-27T07:37:08.023 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-04-27T07:37:08.049 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-27T07:37:08.054 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-27T07:37:08.054 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-27T07:37:08.106 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Ubuntu" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="22.04" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_CODENAME=jammy 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:ID=ubuntu 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE=debian 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-27T07:37:08.107 INFO:teuthology.orchestra.run.smithi183.stdout:UBUNTU_CODENAME=jammy 2024-04-27T07:37:08.108 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-27T07:37:08.132 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T07:37:08.141 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T07:37:08.147 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T07:37:08.147 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-04-27T07:37:08.149 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-04-27T07:37:08.154 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T07:37:08.160 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T07:37:08.160 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-04-27T07:37:08.195 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-04-27T07:37:08.207 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T07:37:08.261 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T07:37:08.276 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T07:37:08.276 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T07:37:08.276 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T07:37:08.277 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T07:37:08.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:37:08.281 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-04-27T07:37:08.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:37:08.282 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-04-27T07:37:08.326 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-56-generic 2024-04-27T07:37:08.327 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.15.0-56-generic 2024-04-27T07:37:08.327 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-04-27T07:37:08.330 INFO:teuthology.orchestra.run.smithi022.stdout:5.15.0-56-generic 2024-04-27T07:37:08.330 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.15.0-56-generic 2024-04-27T07:37:08.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2024-04-27T07:37:08.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-04-27T07:37:08.466 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-04-27T07:37:08.619 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:37:08.619 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-27T07:37:08.622 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-27T07:37:08.641 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:37:08.641 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-27T07:37:08.641 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-27T07:37:08.712 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-27T07:37:08.717 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-27T07:37:08.848 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-27T07:37:08.882 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-27T07:37:08.917 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-27T07:37:08.971 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-27T07:37:08.974 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-27T07:37:08.982 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-27T07:37:08.987 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-27T07:37:08.987 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-27T07:37:08.988 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-27T07:37:09.003 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-27T07:37:09.027 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-27T07:37:09.030 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T07:37:09.030 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-27T07:37:09.051 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-27T07:37:09.059 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-27T07:37:09.061 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-27T07:37:09.064 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-27T07:37:09.090 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-27T07:37:09.104 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-27T07:37:09.104 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-27T07:37:09.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-27T07:37:09.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-27T07:37:09.105 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-27T07:37:09.108 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-27T07:37:09.109 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-27T07:37:09.110 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-27T07:37:09.111 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-27T07:37:09.112 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-27T07:37:09.113 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-27T07:37:09.125 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-27T07:37:09.159 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-27T07:37:09.167 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T07:37:09.167 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-27T07:37:09.190 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-27T07:37:09.200 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-27T07:37:09.204 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T07:37:09.204 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-27T07:37:09.205 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-27T07:37:09.218 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-27T07:37:09.224 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-27T07:37:09.225 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-27T07:37:09.226 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-27T07:37:09.227 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-27T07:37:09.227 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-27T07:37:09.243 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-27T07:37:09.253 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-27T07:37:09.253 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-27T07:37:09.255 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-27T07:37:09.256 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-27T07:37:09.263 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-27T07:37:09.263 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-27T07:37:09.266 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-27T07:37:09.267 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-27T07:37:09.268 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-27T07:37:09.270 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-27T07:37:09.270 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-27T07:37:09.283 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-27T07:37:09.284 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-27T07:37:09.285 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-27T07:37:09.285 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-27T07:37:09.286 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-27T07:37:09.312 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-27T07:37:09.313 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-27T07:37:09.319 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T07:37:09.319 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-27T07:37:09.331 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-27T07:37:09.349 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-27T07:37:09.352 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-27T07:37:09.353 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-27T07:37:09.353 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-27T07:37:09.354 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-27T07:37:09.355 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-27T07:37:09.355 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-27T07:37:09.356 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-27T07:37:09.376 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-27T07:37:09.376 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-27T07:37:09.376 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-27T07:37:09.376 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-27T07:37:09.377 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-27T07:37:09.377 INFO:teuthology.orchestra.run.smithi022.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-27T07:37:14.900 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 13.2 MB in 2s (7,176 kB/s) 2024-04-27T07:37:15.239 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 13.2 MB in 2s (6,627 kB/s) 2024-04-27T07:37:16.195 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:37:16.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T07:37:16.263 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:37:16.440 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-04-27T07:37:16.442 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-04-27T07:37:16.596 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:37:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:37:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:37:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:37:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-27T07:37:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: python2.7-minimal 2024-04-27T07:37:16.597 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:37:16.598 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-04-27T07:37:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T07:37:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-27T07:37:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-27T07:37:16.599 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-04-27T07:37:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-27T07:37:16.599 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2024-04-27T07:37:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: thermald 2024-04-27T07:37:16.648 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-04-27T07:37:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T07:37:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-27T07:37:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-27T07:37:16.649 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2024-04-27T07:37:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-27T07:37:16.666 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:37:16.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T07:37:16.740 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:37:16.783 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T07:37:16.784 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 113 MB of archives. 2024-04-27T07:37:16.784 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-27T07:37:16.784 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-27T07:37:16.943 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-04-27T07:37:16.944 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-04-27T07:37:17.125 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:37:17.125 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:37:17.125 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:37:17.125 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout: python2.7-minimal 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T07:37:17.126 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-27T07:37:17.127 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-27T07:37:17.127 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-04-27T07:37:17.128 INFO:teuthology.orchestra.run.smithi022.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-27T07:37:17.128 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2024-04-27T07:37:17.128 INFO:teuthology.orchestra.run.smithi022.stdout: thermald 2024-04-27T07:37:17.173 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-04-27T07:37:17.173 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T07:37:17.173 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-27T07:37:17.174 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-27T07:37:17.174 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2024-04-27T07:37:17.175 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-27T07:37:17.355 INFO:teuthology.orchestra.run.smithi022.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T07:37:17.355 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 113 MB of archives. 2024-04-27T07:37:17.355 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-27T07:37:17.356 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-27T07:37:18.531 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-27T07:37:18.801 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-27T07:37:18.873 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-27T07:37:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-27T07:37:20.533 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-27T07:37:20.534 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-27T07:37:20.534 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-27T07:37:20.738 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-27T07:37:20.789 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-27T07:37:21.162 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 113 MB in 4s (27.4 MB/s) 2024-04-27T07:37:21.347 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-27T07:37:21.437 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-27T07:37:21.437 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-27T07:37:21.437 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-27T07:37:22.722 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-27T07:37:22.901 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-27T07:37:23.323 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-27T07:37:23.327 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:23.354 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 113 MB in 6s (19.8 MB/s) 2024-04-27T07:37:23.418 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:23.544 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-27T07:37:25.580 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-27T07:37:25.584 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:25.844 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-27T07:37:25.866 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:25.958 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:26.431 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-27T07:37:26.452 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:26.489 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:27.894 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-27T07:37:27.917 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:28.017 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:28.589 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-27T07:37:28.610 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:32.848 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:32.973 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:33.289 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:33.427 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:33.713 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-27T07:37:33.735 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-27T07:37:33.779 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T07:37:35.062 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:35.163 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:35.936 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-27T07:37:35.959 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-27T07:37:36.003 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T07:37:39.062 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-27T07:37:39.099 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:39.137 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:41.289 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-27T07:37:41.326 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T07:37:41.363 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:42.208 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:42.331 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:42.664 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T07:37:42.786 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:42.912 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-27T07:37:43.041 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:44.074 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T07:37:44.410 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:37:44.410 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-27T07:37:44.411 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:37:44.411 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-27T07:37:44.841 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T07:37:44.884 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:45.275 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T07:37:45.371 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:45.497 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-27T07:37:45.625 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:46.312 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:47.005 INFO:teuthology.orchestra.run.smithi022.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:37:47.006 INFO:teuthology.orchestra.run.smithi022.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-27T07:37:47.006 INFO:teuthology.orchestra.run.smithi022.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:37:47.006 INFO:teuthology.orchestra.run.smithi022.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-27T07:37:47.169 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:47.745 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-27T07:37:47.871 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-27T07:37:47.998 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:48.062 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-27T07:37:48.062 INFO:teuthology.orchestra.run.smithi183.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:37:48.613 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:50.473 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-27T07:37:50.923 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-27T07:37:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T07:37:51.123 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-27T07:37:51.123 INFO:teuthology.orchestra.run.smithi022.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:38:02.249 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-27T07:38:02.250 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub' 2024-04-27T07:38:02.260 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T07:38:02.287 INFO:teuthology.orchestra.run.smithi183.stdout:Generating grub configuration file ... 2024-04-27T07:38:02.627 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:38:02.647 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:38:03.017 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:38:03.023 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T07:38:03.090 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T07:38:03.096 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T07:38:03.263 INFO:teuthology.orchestra.run.smithi183.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T07:38:03.263 INFO:teuthology.orchestra.run.smithi183.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T07:38:03.263 INFO:teuthology.orchestra.run.smithi183.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T07:38:03.308 INFO:teuthology.orchestra.run.smithi183.stdout:done 2024-04-27T07:38:03.774 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-04-27T07:38:03.796 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 21 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.105.102 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-04-27T07:38:03.797 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-04-27T07:38:03.798 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-27T07:38:03.798 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T07:38:03.798 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T07:38:03.798 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-27T07:38:03.798 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T07:38:03.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-04-27T07:38:03.923 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-04-27T07:38:04.039 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:38:04.053 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:38:04.055 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:38:04.074 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:38:05.419 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:38:05.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T07:38:05.475 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-27T07:38:05.476 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub' 2024-04-27T07:38:05.495 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:38:05.496 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T07:38:05.519 INFO:teuthology.orchestra.run.smithi022.stdout:Generating grub configuration file ... 2024-04-27T07:38:05.695 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-04-27T07:38:05.695 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-04-27T07:38:05.788 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:38:05.823 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:38:05.870 INFO:teuthology.orchestra.run.smithi183.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-27T07:38:05.871 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:38:05.871 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:38:05.871 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:38:05.871 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:38:05.872 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:38:05.872 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:38:05.872 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:38:05.912 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T07:38:05.914 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 21 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-04-27T07:38:05.931 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.105.102 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-04-27T07:38:05.932 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-04-27T07:38:05.932 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-04-27T07:38:05.977 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.fR3yJr7jSg 2024-04-27T07:38:05.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub/grub.cfg /tmp/tmp.fR3yJr7jSg 2024-04-27T07:38:06.032 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.fR3yJr7jSg 2024-04-27T07:38:06.108 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:38:06.113 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T07:38:06.147 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.fR3yJr7jSg is 10KB 2024-04-27T07:38:06.160 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.fR3yJr7jSg 2024-04-27T07:38:06.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-27T07:38:06.186 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T07:38:06.191 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T07:38:06.221 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:38:06.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-27T07:38:06.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-27T07:38:06.288 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-27T07:38:06.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo update-grub 2024-04-27T07:38:06.356 INFO:teuthology.orchestra.run.smithi022.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T07:38:06.356 INFO:teuthology.orchestra.run.smithi022.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T07:38:06.356 INFO:teuthology.orchestra.run.smithi022.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T07:38:06.390 INFO:teuthology.orchestra.run.smithi022.stdout:done 2024-04-27T07:38:06.857 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2024-04-27T07:38:06.884 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2024-04-27T07:38:06.884 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2024-04-27T07:38:06.884 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2024-04-27T07:38:06.884 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 21 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.15.0.105.102 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2024-04-27T07:38:06.885 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2024-04-27T07:38:06.885 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-27T07:38:06.886 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T07:38:06.886 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T07:38:06.886 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-27T07:38:06.886 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T07:38:06.886 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2024-04-27T07:38:07.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-04-27T07:38:07.181 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:38:07.193 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:38:07.214 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:38:07.303 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:38:07.450 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub' 2024-04-27T07:38:07.452 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T07:38:07.453 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-04-27T07:38:07.711 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:38:07.718 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:38:08.017 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:38:08.022 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T07:38:08.065 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T07:38:08.070 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T07:38:08.209 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T07:38:08.210 INFO:teuthology.orchestra.run.smithi183.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T07:38:08.210 INFO:teuthology.orchestra.run.smithi183.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T07:38:08.231 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-04-27T07:38:08.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-04-27T07:38:08.579 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:38:08.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T07:38:08.649 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:38:08.845 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-04-27T07:38:08.846 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-04-27T07:38:09.012 INFO:teuthology.orchestra.run.smithi022.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:38:09.013 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:38:09.054 INFO:teuthology.orchestra.run.smithi022.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T07:38:09.056 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 21 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.15.0.105.102 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2024-04-27T07:38:09.117 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2024-04-27T07:38:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2024-04-27T07:38:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2024-04-27T07:38:09.118 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-04-27T07:38:09.165 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.n4STgymeMU 2024-04-27T07:38:09.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub/grub.cfg /tmp/tmp.n4STgymeMU 2024-04-27T07:38:09.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.n4STgymeMU 2024-04-27T07:38:09.352 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.n4STgymeMU is 10KB 2024-04-27T07:38:09.365 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.n4STgymeMU 2024-04-27T07:38:09.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-27T07:38:09.430 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:38:09.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-27T07:38:09.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-27T07:38:09.497 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-27T07:38:09.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo update-grub 2024-04-27T07:38:10.664 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub' 2024-04-27T07:38:10.665 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T07:38:10.668 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-04-27T07:38:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T07:38:10.858 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:38:11.076 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T07:38:11.081 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T07:38:11.120 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T07:38:11.125 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T07:38:11.240 INFO:teuthology.orchestra.run.smithi022.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T07:38:11.241 INFO:teuthology.orchestra.run.smithi022.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T07:38:11.241 INFO:teuthology.orchestra.run.smithi022.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T07:38:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-04-27T07:38:11.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-04-27T07:38:38.266 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-27T07:38:38.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:38:38.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:38:41.286 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-27T07:38:41.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:38:41.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:38:56.675 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-27T07:38:59.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-04-27T07:39:05.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:39:05.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:39:08.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-27T07:39:08.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:39:08.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:39:11.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-04-27T07:39:20.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:39:20.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:39:23.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:39:23.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:39:39.167 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-27T07:39:54.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:39:54.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:39:55.158 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-27T07:39:55.567 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-04-27T07:39:56.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:39:56.001 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-27T07:39:56.001 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-04-27T07:39:56.006 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-105-generic 2024-04-27T07:39:56.006 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-27T07:39:56.006 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T07:39:56.006 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-27T07:39:56.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:39:56.154 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-27T07:39:56.154 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-04-27T07:39:56.201 INFO:teuthology.orchestra.run.smithi022.stdout:5.15.0-105-generic 2024-04-27T07:39:56.202 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-27T07:39:56.202 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T07:39:56.202 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-27T07:39:57.008 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-27T07:39:57.009 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T07:39:57.089 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-04-27T07:39:57.108 DEBUG:teuthology.parallel:result is None 2024-04-27T07:39:57.202 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-27T07:39:57.203 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T07:39:57.258 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-04-27T07:39:57.286 DEBUG:teuthology.parallel:result is None 2024-04-27T07:39:57.287 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T07:39:57.295 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T07:39:57.296 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T07:39:57.298 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T07:39:57.304 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T07:39:57.319 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T07:39:57.333 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T07:39:57.333 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T07:39:57.346 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T07:39:57.364 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T07:39:57.375 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T07:39:57.375 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T07:39:57.402 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T07:39:57.417 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T07:39:57.421 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T07:39:57.426 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T07:39:57.431 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T07:39:57.432 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T07:39:57.441 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T07:39:57.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T07:39:57.470 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T07:39:57.490 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T07:39:57.503 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T07:39:57.504 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T07:39:57.526 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T07:39:57.538 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T07:39:57.577 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T07:39:57.625 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:39:57.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T07:39:57.705 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T07:39:57.711 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T07:39:57.759 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:39:57.759 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T07:39:57.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-04-27T07:39:57.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-04-27T07:39:57.905 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T07:39:57.915 INFO:teuthology.task.internal:Starting timer... 2024-04-27T07:39:57.915 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T07:39:57.931 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T07:39:57.944 DEBUG:teuthology.task.selinux:Excluding smithi022: OS 'ubuntu' does not support SELinux 2024-04-27T07:39:57.944 DEBUG:teuthology.task.selinux:Excluding smithi183: OS 'ubuntu' does not support SELinux 2024-04-27T07:39:57.945 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T07:39:57.945 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-27T07:39:57.945 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T07:39:57.945 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T07:39:57.955 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T07:39:57.972 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T07:39:57.973 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T07:47:08.197 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-04-27T07:47:08.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:47:08.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:47:08.282 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-04-27T07:47:08.366 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-27T07:47:08.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:47:08.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:47:08.449 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-27T07:47:08.526 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-27T07:47:08.526 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T07:47:08.538 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T07:47:08.538 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T07:47:08.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:47:08.541 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T07:47:08.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Command line: ntpd -gq 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: ---------------------------------------------------- 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: ntp-4 is maintained by Network Time Foundation, 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: corporation. Support and training for ntp-4 are 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: available at https://www.nwtime.org/support 2024-04-27T07:47:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: ---------------------------------------------------- 2024-04-27T07:47:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: proto: precision = 0.065 usec (-24) 2024-04-27T07:47:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: basedate set to 2022-02-04 2024-04-27T07:47:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: gps base set to 2022-02-06 (week 2196) 2024-04-27T07:47:08.573 INFO:teuthology.orchestra.run.smithi022.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:47:08.573 INFO:teuthology.orchestra.run.smithi022.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen and drop on 0 v6wildcard [::]:123 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen normally on 2 lo 127.0.0.1:123 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen normally on 3 ens1f0 172.21.15.22:123 2024-04-27T07:47:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen normally on 4 lo [::1]:123 2024-04-27T07:47:08.575 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:5de4%4]:123 2024-04-27T07:47:08.575 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:08 ntpd[17137]: Listening on routing socket on fd #22 for interface updates 2024-04-27T07:47:08.595 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-27T07:47:08.595 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Command line: ntpd -gq 2024-04-27T07:47:08.595 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: ---------------------------------------------------- 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: ntp-4 is maintained by Network Time Foundation, 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: corporation. Support and training for ntp-4 are 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: available at https://www.nwtime.org/support 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: ---------------------------------------------------- 2024-04-27T07:47:08.596 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: proto: precision = 0.043 usec (-24) 2024-04-27T07:47:08.597 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: basedate set to 2022-02-04 2024-04-27T07:47:08.597 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: gps base set to 2022-02-06 (week 2196) 2024-04-27T07:47:08.597 INFO:teuthology.orchestra.run.smithi183.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:47:08.597 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:47:08.597 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:47:08.598 INFO:teuthology.orchestra.run.smithi183.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen and drop on 0 v6wildcard [::]:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen normally on 2 lo 127.0.0.1:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen normally on 3 enp3s0f1 172.21.15.183:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen normally on 4 lo [::1]:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8085%5]:123 2024-04-27T07:47:08.599 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:08 ntpd[17233]: Listening on routing socket on fd #22 for interface updates 2024-04-27T07:47:09.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:09 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:09.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:09 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:09.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:09 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:09.598 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:09 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:10.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:10 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:10.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:10 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:11 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:11 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:12.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:12 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:12.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:12.598 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:12.598 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:12 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:13.574 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.574 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.574 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.574 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.575 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.575 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.575 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:13.576 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:13.576 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.576 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.576 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:13.576 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:13 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:13.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:13 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:14.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:14.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:14 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:14.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:14 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:15.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:15.573 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:15 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:15.596 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:15 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:16.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.571 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stdout:27 Apr 07:47:16 ntpd[17137]: ntpd: time slew +0.000105 s 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stdout:ntpd: time slew +0.000105s 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-27T07:47:16.572 INFO:teuthology.orchestra.run.smithi022.stderr:27 Apr 07:47:16 ntpd[17137]: can't open /var/log/ntpstats/loopstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:47:16.597 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:47:16.598 INFO:teuthology.orchestra.run.smithi183.stdout:27 Apr 07:47:16 ntpd[17233]: ntpd: time slew -0.002732 s 2024-04-27T07:47:16.598 INFO:teuthology.orchestra.run.smithi183.stdout:ntpd: time slew -0.002732s 2024-04-27T07:47:16.599 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-27T07:47:16.599 INFO:teuthology.orchestra.run.smithi183.stderr:27 Apr 07:47:16 ntpd[17233]: can't open /var/log/ntpstats/loopstats.20240427: Permission denied 2024-04-27T07:47:16.626 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:47:16.626 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2024-04-27T07:47:16.626 INFO:teuthology.orchestra.run.smithi022.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.627 INFO:teuthology.orchestra.run.smithi022.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.627 INFO:teuthology.orchestra.run.smithi022.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.627 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:47:16.647 INFO:teuthology.run_tasks:Running task install... 2024-04-27T07:47:16.656 DEBUG:teuthology.task.install:project ceph 2024-04-27T07:47:16.656 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}} 2024-04-27T07:47:16.656 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'} 2024-04-27T07:47:16.656 INFO:teuthology.task.install:Using flavor: default 2024-04-27T07:47:16.662 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T07:47:16.662 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-27T07:47:16.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-key list | grep Ceph 2024-04-27T07:47:16.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2024-04-27T07:47:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-27T07:47:16.738 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-27T07:47:16.763 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-27T07:47:16.763 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph.com (release key) 2024-04-27T07:47:16.764 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-27T07:47:16.764 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-27T07:47:16.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:47:16.769 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-27T07:47:16.770 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2024-04-27T07:47:16.770 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-27T07:47:16.770 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-27T07:47:16.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:47:16.910 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default/ 2024-04-27T07:47:16.911 INFO:teuthology.task.install.deb:Package version is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:47:16.990 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default/ 2024-04-27T07:47:16.991 INFO:teuthology.task.install.deb:Package version is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:47:17.114 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:47:17.114 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-27T07:47:17.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-04-27T07:47:17.262 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:47:17.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-27T07:47:17.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-04-27T07:47:17.326 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:47:17.403 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy InRelease 2024-04-27T07:47:17.441 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:47:17.449 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-27T07:47:17.455 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:47:17.478 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-27T07:47:17.527 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:47:17.542 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-04-27T07:47:17.568 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:47:17.590 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-27T07:47:17.608 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:47:17.649 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:47:17.659 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy InRelease 2024-04-27T07:47:17.697 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-27T07:47:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:47:17.738 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-27T07:47:17.753 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 29.3 kB in 0s (66.4 kB/s) 2024-04-27T07:47:17.774 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-04-27T07:47:17.818 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-27T07:47:17.904 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 29.3 kB in 0s (67.1 kB/s) 2024-04-27T07:47:18.750 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:47:18.766 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3227-gc37f63b3-1jammy cephadm=19.0.0-3227-gc37f63b3-1jammy ceph-mds=19.0.0-3227-gc37f63b3-1jammy ceph-mgr=19.0.0-3227-gc37f63b3-1jammy ceph-common=19.0.0-3227-gc37f63b3-1jammy ceph-fuse=19.0.0-3227-gc37f63b3-1jammy ceph-test=19.0.0-3227-gc37f63b3-1jammy ceph-volume=19.0.0-3227-gc37f63b3-1jammy radosgw=19.0.0-3227-gc37f63b3-1jammy python3-rados=19.0.0-3227-gc37f63b3-1jammy python3-rgw=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy python3-rbd=19.0.0-3227-gc37f63b3-1jammy libcephfs2=19.0.0-3227-gc37f63b3-1jammy libcephfs-dev=19.0.0-3227-gc37f63b3-1jammy librados2=19.0.0-3227-gc37f63b3-1jammy librbd1=19.0.0-3227-gc37f63b3-1jammy rbd-fuse=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy cephfs-shell=19.0.0-3227-gc37f63b3-1jammy cephfs-top=19.0.0-3227-gc37f63b3-1jammy cephfs-mirror=19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:47:18.826 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:47:18.894 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:47:18.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3227-gc37f63b3-1jammy cephadm=19.0.0-3227-gc37f63b3-1jammy ceph-mds=19.0.0-3227-gc37f63b3-1jammy ceph-mgr=19.0.0-3227-gc37f63b3-1jammy ceph-common=19.0.0-3227-gc37f63b3-1jammy ceph-fuse=19.0.0-3227-gc37f63b3-1jammy ceph-test=19.0.0-3227-gc37f63b3-1jammy ceph-volume=19.0.0-3227-gc37f63b3-1jammy radosgw=19.0.0-3227-gc37f63b3-1jammy python3-rados=19.0.0-3227-gc37f63b3-1jammy python3-rgw=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy python3-rbd=19.0.0-3227-gc37f63b3-1jammy libcephfs2=19.0.0-3227-gc37f63b3-1jammy libcephfs-dev=19.0.0-3227-gc37f63b3-1jammy librados2=19.0.0-3227-gc37f63b3-1jammy librbd1=19.0.0-3227-gc37f63b3-1jammy rbd-fuse=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy cephfs-shell=19.0.0-3227-gc37f63b3-1jammy cephfs-top=19.0.0-3227-gc37f63b3-1jammy cephfs-mirror=19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:47:18.970 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:47:19.025 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-04-27T07:47:19.026 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-04-27T07:47:19.170 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-04-27T07:47:19.171 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-04-27T07:47:19.209 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:47:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:47:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:47:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:47:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:47:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:47:19.210 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:47:19.211 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-04-27T07:47:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-27T07:47:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-27T07:47:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-27T07:47:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-27T07:47:19.215 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2024-04-27T07:47:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: btrfs-tools 2024-04-27T07:47:19.318 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-04-27T07:47:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-27T07:47:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-27T07:47:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-27T07:47:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-27T07:47:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-27T07:47:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-27T07:47:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-27T07:47:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-27T07:47:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: socat unzip xmlstarlet zip 2024-04-27T07:47:19.370 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:47:19.370 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:47:19.370 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:47:19.370 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:47:19.371 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:47:19.371 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:47:19.371 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:47:19.372 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-04-27T07:47:19.372 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-27T07:47:19.372 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-27T07:47:19.372 INFO:teuthology.orchestra.run.smithi022.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:47:19.372 INFO:teuthology.orchestra.run.smithi022.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-27T07:47:19.373 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-27T07:47:19.373 INFO:teuthology.orchestra.run.smithi022.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-27T07:47:19.373 INFO:teuthology.orchestra.run.smithi022.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-27T07:47:19.373 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-27T07:47:19.374 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-27T07:47:19.375 INFO:teuthology.orchestra.run.smithi022.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-27T07:47:19.376 INFO:teuthology.orchestra.run.smithi022.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-27T07:47:19.376 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2024-04-27T07:47:19.376 INFO:teuthology.orchestra.run.smithi022.stdout: btrfs-tools 2024-04-27T07:47:19.386 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-27T07:47:19.387 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 182 MB of archives. 2024-04-27T07:47:19.387 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-27T07:47:19.387 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-27T07:47:19.408 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-27T07:47:19.421 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-27T07:47:19.463 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3227-gc37f63b3-1jammy [3,556 kB] 2024-04-27T07:47:19.470 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-27T07:47:19.488 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-04-27T07:47:19.488 INFO:teuthology.orchestra.run.smithi022.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-27T07:47:19.488 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-27T07:47:19.488 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-27T07:47:19.488 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-27T07:47:19.489 INFO:teuthology.orchestra.run.smithi022.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-27T07:47:19.490 INFO:teuthology.orchestra.run.smithi022.stdout: socat unzip xmlstarlet zip 2024-04-27T07:47:19.496 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-27T07:47:19.499 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-27T07:47:19.514 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-27T07:47:19.573 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-27T07:47:19.577 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-27T07:47:19.589 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-27T07:47:19.592 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-27T07:47:19.593 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-27T07:47:19.600 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-27T07:47:19.602 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-27T07:47:19.604 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-27T07:47:19.606 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-27T07:47:19.606 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-27T07:47:19.619 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-27T07:47:19.621 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-27T07:47:19.623 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-27T07:47:19.624 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-27T07:47:19.633 INFO:teuthology.orchestra.run.smithi022.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-27T07:47:19.633 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 182 MB of archives. 2024-04-27T07:47:19.633 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-27T07:47:19.633 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3227-gc37f63b3-1jammy [3,556 kB] 2024-04-27T07:47:19.640 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-27T07:47:19.641 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-27T07:47:19.641 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-27T07:47:19.641 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-27T07:47:19.642 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-27T07:47:19.642 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-27T07:47:19.660 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-27T07:47:19.661 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-27T07:47:19.663 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-27T07:47:19.664 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-27T07:47:19.678 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-27T07:47:19.680 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-27T07:47:19.681 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-27T07:47:19.682 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-27T07:47:19.682 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-27T07:47:19.683 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-27T07:47:19.684 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-27T07:47:19.702 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-27T07:47:19.703 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-27T07:47:19.717 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-27T07:47:19.717 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-27T07:47:19.723 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-27T07:47:19.724 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-27T07:47:19.726 INFO:teuthology.orchestra.run.smithi183.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-27T07:47:19.730 INFO:teuthology.orchestra.run.smithi183.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-27T07:47:19.731 INFO:teuthology.orchestra.run.smithi183.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-27T07:47:19.736 INFO:teuthology.orchestra.run.smithi183.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-27T07:47:19.743 INFO:teuthology.orchestra.run.smithi183.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-27T07:47:19.747 INFO:teuthology.orchestra.run.smithi183.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-27T07:47:19.748 INFO:teuthology.orchestra.run.smithi183.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-27T07:47:19.749 INFO:teuthology.orchestra.run.smithi183.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-27T07:47:19.761 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-27T07:47:19.764 INFO:teuthology.orchestra.run.smithi183.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-27T07:47:19.807 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-27T07:47:19.833 INFO:teuthology.orchestra.run.smithi183.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-27T07:47:19.834 INFO:teuthology.orchestra.run.smithi183.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-27T07:47:19.836 INFO:teuthology.orchestra.run.smithi183.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-27T07:47:19.836 INFO:teuthology.orchestra.run.smithi183.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-27T07:47:19.837 INFO:teuthology.orchestra.run.smithi183.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-27T07:47:19.845 INFO:teuthology.orchestra.run.smithi183.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3227-gc37f63b3-1jammy [3,224 kB] 2024-04-27T07:47:19.854 INFO:teuthology.orchestra.run.smithi183.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-27T07:47:19.856 INFO:teuthology.orchestra.run.smithi183.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-27T07:47:19.856 INFO:teuthology.orchestra.run.smithi183.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-27T07:47:19.871 INFO:teuthology.orchestra.run.smithi183.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-27T07:47:19.871 INFO:teuthology.orchestra.run.smithi183.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-27T07:47:19.877 INFO:teuthology.orchestra.run.smithi183.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-27T07:47:19.877 INFO:teuthology.orchestra.run.smithi183.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-27T07:47:19.878 INFO:teuthology.orchestra.run.smithi183.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-27T07:47:19.878 INFO:teuthology.orchestra.run.smithi183.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-27T07:47:19.879 INFO:teuthology.orchestra.run.smithi183.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-27T07:47:19.898 INFO:teuthology.orchestra.run.smithi183.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-27T07:47:19.901 INFO:teuthology.orchestra.run.smithi183.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-27T07:47:19.902 INFO:teuthology.orchestra.run.smithi183.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-27T07:47:19.903 INFO:teuthology.orchestra.run.smithi183.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-27T07:47:19.904 INFO:teuthology.orchestra.run.smithi183.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-27T07:47:19.924 INFO:teuthology.orchestra.run.smithi183.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-27T07:47:19.925 INFO:teuthology.orchestra.run.smithi183.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-27T07:47:19.925 INFO:teuthology.orchestra.run.smithi183.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-27T07:47:19.925 INFO:teuthology.orchestra.run.smithi183.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-27T07:47:19.925 INFO:teuthology.orchestra.run.smithi183.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-27T07:47:19.942 INFO:teuthology.orchestra.run.smithi183.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-27T07:47:19.942 INFO:teuthology.orchestra.run.smithi183.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-27T07:47:19.945 INFO:teuthology.orchestra.run.smithi183.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-27T07:47:19.945 INFO:teuthology.orchestra.run.smithi183.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-27T07:47:19.946 INFO:teuthology.orchestra.run.smithi183.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-27T07:47:19.962 INFO:teuthology.orchestra.run.smithi183.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-27T07:47:19.964 INFO:teuthology.orchestra.run.smithi183.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-27T07:47:19.964 INFO:teuthology.orchestra.run.smithi183.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-27T07:47:19.971 INFO:teuthology.orchestra.run.smithi183.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-27T07:47:19.972 INFO:teuthology.orchestra.run.smithi183.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-27T07:47:19.983 INFO:teuthology.orchestra.run.smithi183.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-27T07:47:19.985 INFO:teuthology.orchestra.run.smithi183.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-27T07:47:19.985 INFO:teuthology.orchestra.run.smithi183.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-27T07:47:20.002 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-27T07:47:20.020 INFO:teuthology.orchestra.run.smithi183.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3227-gc37f63b3-1jammy [712 kB] 2024-04-27T07:47:20.029 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3227-gc37f63b3-1jammy [3,224 kB] 2024-04-27T07:47:20.059 INFO:teuthology.orchestra.run.smithi183.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3227-gc37f63b3-1jammy [356 kB] 2024-04-27T07:47:20.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3227-gc37f63b3-1jammy [32.8 kB] 2024-04-27T07:47:20.072 INFO:teuthology.orchestra.run.smithi183.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3227-gc37f63b3-1jammy [180 kB] 2024-04-27T07:47:20.089 INFO:teuthology.orchestra.run.smithi183.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3227-gc37f63b3-1jammy [67.1 kB] 2024-04-27T07:47:20.091 INFO:teuthology.orchestra.run.smithi183.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3227-gc37f63b3-1jammy [330 kB] 2024-04-27T07:47:20.100 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-27T07:47:20.113 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-27T07:47:20.115 INFO:teuthology.orchestra.run.smithi183.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3227-gc37f63b3-1jammy [6,893 kB] 2024-04-27T07:47:20.167 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-27T07:47:20.273 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3227-gc37f63b3-1jammy [712 kB] 2024-04-27T07:47:20.319 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3227-gc37f63b3-1jammy [356 kB] 2024-04-27T07:47:20.345 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-27T07:47:20.354 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-27T07:47:20.354 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3227-gc37f63b3-1jammy [32.8 kB] 2024-04-27T07:47:20.355 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3227-gc37f63b3-1jammy [180 kB] 2024-04-27T07:47:20.361 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3227-gc37f63b3-1jammy [67.1 kB] 2024-04-27T07:47:20.364 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3227-gc37f63b3-1jammy [330 kB] 2024-04-27T07:47:20.382 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-27T07:47:20.391 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-27T07:47:20.399 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3227-gc37f63b3-1jammy [6,893 kB] 2024-04-27T07:47:20.506 INFO:teuthology.orchestra.run.smithi183.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3227-gc37f63b3-1jammy [112 kB] 2024-04-27T07:47:20.509 INFO:teuthology.orchestra.run.smithi183.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3227-gc37f63b3-1jammy [466 kB] 2024-04-27T07:47:20.524 INFO:teuthology.orchestra.run.smithi183.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3227-gc37f63b3-1jammy [25.8 MB] 2024-04-27T07:47:20.557 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-27T07:47:20.960 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3227-gc37f63b3-1jammy [112 kB] 2024-04-27T07:47:20.962 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3227-gc37f63b3-1jammy [466 kB] 2024-04-27T07:47:20.977 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3227-gc37f63b3-1jammy [25.8 MB] 2024-04-27T07:47:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-27T07:47:21.142 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-27T07:47:21.158 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-27T07:47:21.166 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-27T07:47:21.167 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-27T07:47:21.183 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-27T07:47:21.221 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-27T07:47:21.235 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-27T07:47:21.239 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-27T07:47:21.239 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-27T07:47:21.411 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-27T07:47:21.494 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-27T07:47:21.500 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-27T07:47:21.576 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-27T07:47:21.602 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-27T07:47:21.618 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-27T07:47:21.635 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-27T07:47:21.823 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-27T07:47:21.839 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-27T07:47:21.846 INFO:teuthology.orchestra.run.smithi022.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-27T07:47:21.852 INFO:teuthology.orchestra.run.smithi022.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-27T07:47:22.036 INFO:teuthology.orchestra.run.smithi022.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-27T07:47:22.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3227-gc37f63b3-1jammy [5,092 kB] 2024-04-27T07:47:22.119 INFO:teuthology.orchestra.run.smithi022.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-27T07:47:22.133 INFO:teuthology.orchestra.run.smithi022.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-27T07:47:22.297 INFO:teuthology.orchestra.run.smithi183.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3227-gc37f63b3-1jammy [244 kB] 2024-04-27T07:47:22.308 INFO:teuthology.orchestra.run.smithi022.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-27T07:47:22.309 INFO:teuthology.orchestra.run.smithi183.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:47:22.313 INFO:teuthology.orchestra.run.smithi183.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3227-gc37f63b3-1jammy [1,526 kB] 2024-04-27T07:47:22.354 INFO:teuthology.orchestra.run.smithi022.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-27T07:47:22.367 INFO:teuthology.orchestra.run.smithi022.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-27T07:47:22.391 INFO:teuthology.orchestra.run.smithi183.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3227-gc37f63b3-1jammy [6,257 kB] 2024-04-27T07:47:22.528 INFO:teuthology.orchestra.run.smithi022.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-27T07:47:22.529 INFO:teuthology.orchestra.run.smithi022.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-27T07:47:22.533 INFO:teuthology.orchestra.run.smithi022.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-27T07:47:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-27T07:47:22.552 INFO:teuthology.orchestra.run.smithi022.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-27T07:47:22.736 INFO:teuthology.orchestra.run.smithi022.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-27T07:47:22.762 INFO:teuthology.orchestra.run.smithi183.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3227-gc37f63b3-1jammy [22.8 MB] 2024-04-27T07:47:23.000 INFO:teuthology.orchestra.run.smithi022.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-27T07:47:23.088 INFO:teuthology.orchestra.run.smithi022.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3227-gc37f63b3-1jammy [5,092 kB] 2024-04-27T07:47:23.116 INFO:teuthology.orchestra.run.smithi022.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-27T07:47:23.159 INFO:teuthology.orchestra.run.smithi022.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-27T07:47:23.242 INFO:teuthology.orchestra.run.smithi022.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-27T07:47:23.254 INFO:teuthology.orchestra.run.smithi022.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-27T07:47:23.267 INFO:teuthology.orchestra.run.smithi022.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-27T07:47:23.307 INFO:teuthology.orchestra.run.smithi022.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-27T07:47:23.405 INFO:teuthology.orchestra.run.smithi022.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3227-gc37f63b3-1jammy [244 kB] 2024-04-27T07:47:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:47:23.415 INFO:teuthology.orchestra.run.smithi022.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3227-gc37f63b3-1jammy [1,526 kB] 2024-04-27T07:47:23.551 INFO:teuthology.orchestra.run.smithi022.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3227-gc37f63b3-1jammy [6,257 kB] 2024-04-27T07:47:23.568 INFO:teuthology.orchestra.run.smithi022.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-27T07:47:23.570 INFO:teuthology.orchestra.run.smithi022.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-27T07:47:23.572 INFO:teuthology.orchestra.run.smithi022.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-27T07:47:23.737 INFO:teuthology.orchestra.run.smithi022.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-27T07:47:24.040 INFO:teuthology.orchestra.run.smithi022.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3227-gc37f63b3-1jammy [22.8 MB] 2024-04-27T07:47:24.066 INFO:teuthology.orchestra.run.smithi022.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-27T07:47:24.128 INFO:teuthology.orchestra.run.smithi183.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3227-gc37f63b3-1jammy [14.1 kB] 2024-04-27T07:47:24.129 INFO:teuthology.orchestra.run.smithi183.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [870 kB] 2024-04-27T07:47:24.145 INFO:teuthology.orchestra.run.smithi183.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3227-gc37f63b3-1jammy [2,437 kB] 2024-04-27T07:47:24.209 INFO:teuthology.orchestra.run.smithi183.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3227-gc37f63b3-1jammy [791 kB] 2024-04-27T07:47:24.267 INFO:teuthology.orchestra.run.smithi183.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3227-gc37f63b3-1jammy [148 kB] 2024-04-27T07:47:24.271 INFO:teuthology.orchestra.run.smithi183.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3227-gc37f63b3-1jammy [2,297 kB] 2024-04-27T07:47:24.475 INFO:teuthology.orchestra.run.smithi022.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-27T07:47:24.482 INFO:teuthology.orchestra.run.smithi183.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3227-gc37f63b3-1jammy [8,625 kB] 2024-04-27T07:47:24.484 INFO:teuthology.orchestra.run.smithi022.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-27T07:47:24.486 INFO:teuthology.orchestra.run.smithi022.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-27T07:47:24.557 INFO:teuthology.orchestra.run.smithi022.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-27T07:47:24.557 INFO:teuthology.orchestra.run.smithi022.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-27T07:47:24.559 INFO:teuthology.orchestra.run.smithi022.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-27T07:47:24.560 INFO:teuthology.orchestra.run.smithi022.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-27T07:47:24.561 INFO:teuthology.orchestra.run.smithi022.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-27T07:47:24.727 INFO:teuthology.orchestra.run.smithi022.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-27T07:47:24.815 INFO:teuthology.orchestra.run.smithi022.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-27T07:47:24.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-27T07:47:25.043 INFO:teuthology.orchestra.run.smithi183.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3227-gc37f63b3-1jammy [14.2 kB] 2024-04-27T07:47:25.043 INFO:teuthology.orchestra.run.smithi183.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3227-gc37f63b3-1jammy [51.6 MB] 2024-04-27T07:47:25.157 INFO:teuthology.orchestra.run.smithi022.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-27T07:47:25.200 INFO:teuthology.orchestra.run.smithi022.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-27T07:47:25.224 INFO:teuthology.orchestra.run.smithi022.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-27T07:47:25.232 INFO:teuthology.orchestra.run.smithi022.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-27T07:47:25.278 INFO:teuthology.orchestra.run.smithi022.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-27T07:47:25.306 INFO:teuthology.orchestra.run.smithi022.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-27T07:47:25.307 INFO:teuthology.orchestra.run.smithi022.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-27T07:47:25.312 INFO:teuthology.orchestra.run.smithi022.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-27T07:47:25.480 INFO:teuthology.orchestra.run.smithi022.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-27T07:47:26.013 INFO:teuthology.orchestra.run.smithi022.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-27T07:47:26.031 INFO:teuthology.orchestra.run.smithi022.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-27T07:47:26.093 INFO:teuthology.orchestra.run.smithi022.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-27T07:47:26.093 INFO:teuthology.orchestra.run.smithi022.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-27T07:47:26.130 INFO:teuthology.orchestra.run.smithi022.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-27T07:47:26.130 INFO:teuthology.orchestra.run.smithi022.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3227-gc37f63b3-1jammy [14.1 kB] 2024-04-27T07:47:26.130 INFO:teuthology.orchestra.run.smithi022.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-27T07:47:26.130 INFO:teuthology.orchestra.run.smithi022.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [870 kB] 2024-04-27T07:47:26.134 INFO:teuthology.orchestra.run.smithi022.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-27T07:47:26.136 INFO:teuthology.orchestra.run.smithi022.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-27T07:47:26.148 INFO:teuthology.orchestra.run.smithi022.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3227-gc37f63b3-1jammy [2,437 kB] 2024-04-27T07:47:26.215 INFO:teuthology.orchestra.run.smithi022.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-27T07:47:26.216 INFO:teuthology.orchestra.run.smithi022.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-27T07:47:26.307 INFO:teuthology.orchestra.run.smithi022.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3227-gc37f63b3-1jammy [791 kB] 2024-04-27T07:47:26.382 INFO:teuthology.orchestra.run.smithi022.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3227-gc37f63b3-1jammy [148 kB] 2024-04-27T07:47:26.384 INFO:teuthology.orchestra.run.smithi022.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-27T07:47:26.386 INFO:teuthology.orchestra.run.smithi022.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3227-gc37f63b3-1jammy [2,297 kB] 2024-04-27T07:47:26.563 INFO:teuthology.orchestra.run.smithi022.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3227-gc37f63b3-1jammy [8,625 kB] 2024-04-27T07:47:26.833 INFO:teuthology.orchestra.run.smithi022.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-27T07:47:26.848 INFO:teuthology.orchestra.run.smithi022.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-27T07:47:27.229 INFO:teuthology.orchestra.run.smithi022.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3227-gc37f63b3-1jammy [14.2 kB] 2024-04-27T07:47:27.229 INFO:teuthology.orchestra.run.smithi022.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3227-gc37f63b3-1jammy [51.6 MB] 2024-04-27T07:47:28.282 INFO:teuthology.orchestra.run.smithi183.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:47:28.284 INFO:teuthology.orchestra.run.smithi183.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3227-gc37f63b3-1jammy [232 kB] 2024-04-27T07:47:28.289 INFO:teuthology.orchestra.run.smithi183.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3227-gc37f63b3-1jammy [33.9 kB] 2024-04-27T07:47:28.289 INFO:teuthology.orchestra.run.smithi183.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3227-gc37f63b3-1jammy [23.4 kB] 2024-04-27T07:47:28.290 INFO:teuthology.orchestra.run.smithi183.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3227-gc37f63b3-1jammy [39.0 kB] 2024-04-27T07:47:28.291 INFO:teuthology.orchestra.run.smithi183.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3227-gc37f63b3-1jammy [13.6 MB] 2024-04-27T07:47:28.944 INFO:teuthology.orchestra.run.smithi183.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [91.4 kB] 2024-04-27T07:47:29.558 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 182 MB in 10s (18.9 MB/s) 2024-04-27T07:47:29.691 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-27T07:47:29.790 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-27T07:47:29.795 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-27T07:47:29.839 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-27T07:47:30.151 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-27T07:47:30.177 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-27T07:47:30.186 INFO:teuthology.orchestra.run.smithi022.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:47:30.188 INFO:teuthology.orchestra.run.smithi022.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3227-gc37f63b3-1jammy [232 kB] 2024-04-27T07:47:30.193 INFO:teuthology.orchestra.run.smithi022.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3227-gc37f63b3-1jammy [33.9 kB] 2024-04-27T07:47:30.194 INFO:teuthology.orchestra.run.smithi022.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3227-gc37f63b3-1jammy [23.4 kB] 2024-04-27T07:47:30.195 INFO:teuthology.orchestra.run.smithi022.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3227-gc37f63b3-1jammy [39.0 kB] 2024-04-27T07:47:30.196 INFO:teuthology.orchestra.run.smithi022.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3227-gc37f63b3-1jammy [13.6 MB] 2024-04-27T07:47:30.217 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:47:30.536 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cmd2. 2024-04-27T07:47:30.563 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-27T07:47:30.591 INFO:teuthology.orchestra.run.smithi022.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [91.4 kB] 2024-04-27T07:47:30.603 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-27T07:47:30.948 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-27T07:47:30.974 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-27T07:47:31.056 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:47:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 182 MB in 11s (16.4 MB/s) 2024-04-27T07:47:31.379 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-27T07:47:31.410 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-27T07:47:31.436 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-27T07:47:31.477 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:47:31.481 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-27T07:47:31.487 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-27T07:47:31.527 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-27T07:47:31.830 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-27T07:47:31.856 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-27T07:47:31.896 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:47:32.054 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-27T07:47:32.081 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-27T07:47:32.129 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:47:32.191 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cmd2. 2024-04-27T07:47:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-27T07:47:32.241 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-27T07:47:32.535 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-27T07:47:32.562 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-27T07:47:32.635 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:47:32.674 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-27T07:47:32.701 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:32.749 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:32.964 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-27T07:47:32.991 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-27T07:47:33.030 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:47:33.169 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-27T07:47:33.195 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:33.237 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:33.350 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-27T07:47:33.377 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-27T07:47:33.417 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:47:33.647 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-27T07:47:33.673 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:33.722 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:33.803 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-27T07:47:33.830 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:33.870 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:34.108 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-27T07:47:34.135 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-27T07:47:34.175 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:47:34.256 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-27T07:47:34.283 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:34.331 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:34.503 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librados2. 2024-04-27T07:47:34.529 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../010-librados2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:34.569 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:34.676 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-27T07:47:34.703 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:47:34.743 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:47:35.045 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-27T07:47:35.061 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-27T07:47:35.095 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:47:35.097 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librbd1. 2024-04-27T07:47:35.123 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../011-librbd1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:35.172 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librados2. 2024-04-27T07:47:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../010-librados2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:35.683 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs2. 2024-04-27T07:47:35.710 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:35.758 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:36.044 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rados. 2024-04-27T07:47:36.060 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:36.102 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:36.168 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librbd1. 2024-04-27T07:47:36.196 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../011-librbd1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:36.430 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-27T07:47:36.456 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:36.468 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:36.505 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:36.808 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cephfs. 2024-04-27T07:47:36.835 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:36.882 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:36.996 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs2. 2024-04-27T07:47:37.022 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:37.062 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:37.269 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-27T07:47:37.289 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rados. 2024-04-27T07:47:37.295 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:37.305 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:37.339 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:37.343 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:37.644 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-27T07:47:37.671 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:37.705 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-prettytable. 2024-04-27T07:47:37.711 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:37.731 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-27T07:47:37.771 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-27T07:47:37.946 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cephfs. 2024-04-27T07:47:37.962 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:38.005 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:38.032 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rbd. 2024-04-27T07:47:38.047 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:38.099 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:38.274 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-27T07:47:38.290 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:38.510 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-27T07:47:38.537 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-27T07:47:38.585 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-prettytable. 2024-04-27T07:47:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-27T07:47:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-27T07:47:38.913 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-27T07:47:38.940 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-27T07:47:38.954 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rbd. 2024-04-27T07:47:38.971 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:38.987 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:47:39.012 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:39.349 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-27T07:47:39.365 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-27T07:47:39.376 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-27T07:47:39.392 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-27T07:47:39.424 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:47:39.432 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:47:39.710 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-27T07:47:39.719 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua5.1. 2024-04-27T07:47:39.734 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-27T07:47:39.736 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-27T07:47:39.776 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:47:39.777 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:47:40.045 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-27T07:47:40.063 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-27T07:47:40.097 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-any. 2024-04-27T07:47:40.104 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:47:40.124 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-27T07:47:40.163 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-27T07:47:40.390 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package zip. 2024-04-27T07:47:40.406 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-27T07:47:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua5.1. 2024-04-27T07:47:40.441 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking zip (3.0-12build2) ... 2024-04-27T07:47:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-27T07:47:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:47:40.746 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package unzip. 2024-04-27T07:47:40.850 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-any. 2024-04-27T07:47:40.850 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-27T07:47:40.850 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-27T07:47:40.850 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:47:40.885 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-27T07:47:41.129 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package zip. 2024-04-27T07:47:41.156 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-27T07:47:41.189 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package luarocks. 2024-04-27T07:47:41.196 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking zip (3.0-12build2) ... 2024-04-27T07:47:41.215 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-27T07:47:41.256 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:47:41.473 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package unzip. 2024-04-27T07:47:41.500 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:47:41.638 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librgw2. 2024-04-27T07:47:41.665 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../027-librgw2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:41.705 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:41.927 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package luarocks. 2024-04-27T07:47:41.954 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-27T07:47:41.994 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:47:42.385 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librgw2. 2024-04-27T07:47:42.412 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../027-librgw2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:42.432 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rgw. 2024-04-27T07:47:42.451 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:42.458 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:42.498 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:42.810 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-27T07:47:42.837 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-27T07:47:42.877 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:47:43.012 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rgw. 2024-04-27T07:47:43.028 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:43.070 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:43.213 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libradosstriper1. 2024-04-27T07:47:43.239 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:43.279 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:43.356 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-27T07:47:43.372 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-27T07:47:43.406 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:47:43.686 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libradosstriper1. 2024-04-27T07:47:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:43.716 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-common. 2024-04-27T07:47:43.742 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:43.742 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:43.782 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:44.095 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-common. 2024-04-27T07:47:44.122 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:44.162 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:45.179 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-base. 2024-04-27T07:47:45.205 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:45.272 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:45.492 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-base. 2024-04-27T07:47:45.519 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:45.560 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:45.858 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-27T07:47:45.885 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-27T07:47:45.925 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:47:46.162 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-27T07:47:46.190 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-27T07:47:46.211 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cheroot. 2024-04-27T07:47:46.229 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:47:46.237 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-27T07:47:46.269 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:47:46.507 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cheroot. 2024-04-27T07:47:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-27T07:47:46.539 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-27T07:47:46.551 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-27T07:47:46.573 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:47:46.589 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:47:46.875 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-27T07:47:46.901 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-27T07:47:46.942 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:47:47.051 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-27T07:47:47.078 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-27T07:47:47.109 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:47:47.244 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-27T07:47:47.270 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-27T07:47:47.328 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:47:47.404 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-27T07:47:47.431 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-27T07:47:47.470 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:47:47.588 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempora. 2024-04-27T07:47:47.615 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-27T07:47:47.647 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-27T07:47:47.773 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-27T07:47:47.800 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-27T07:47:47.840 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:47:47.933 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-portend. 2024-04-27T07:47:47.960 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-27T07:47:48.000 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-27T07:47:48.084 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-tempora. 2024-04-27T07:47:48.100 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-27T07:47:48.142 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-27T07:47:48.302 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-27T07:47:48.329 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-27T07:47:48.369 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-27T07:47:48.428 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-portend. 2024-04-27T07:47:48.456 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-27T07:47:48.495 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-27T07:47:48.655 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-27T07:47:48.682 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-27T07:47:48.721 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:47:48.798 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-27T07:47:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-27T07:47:48.864 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-27T07:47:49.025 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-natsort. 2024-04-27T07:47:49.052 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-27T07:47:49.092 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-27T07:47:49.133 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-27T07:47:49.150 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-27T07:47:49.183 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:47:49.403 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-logutils. 2024-04-27T07:47:49.432 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-27T07:47:49.469 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-27T07:47:49.529 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-natsort. 2024-04-27T07:47:49.556 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-27T07:47:49.654 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-27T07:47:49.781 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-27T07:47:49.807 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-27T07:47:49.847 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:47:49.948 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-logutils. 2024-04-27T07:47:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-27T07:47:50.015 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-27T07:47:50.125 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-mako. 2024-04-27T07:47:50.151 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-27T07:47:50.183 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:47:50.260 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-27T07:47:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-27T07:47:50.326 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:47:50.470 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-27T07:47:50.496 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-27T07:47:50.528 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:47:50.629 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-mako. 2024-04-27T07:47:50.656 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-27T07:47:50.695 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:47:50.831 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-27T07:47:50.857 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-27T07:47:50.897 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:47:51.032 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-27T07:47:51.059 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-27T07:47:51.098 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:47:51.200 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webob. 2024-04-27T07:47:51.227 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-27T07:47:51.267 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:47:51.409 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-27T07:47:51.436 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-27T07:47:51.475 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:47:51.644 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-waitress. 2024-04-27T07:47:51.671 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-27T07:47:51.712 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webob. 2024-04-27T07:47:51.714 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:47:51.728 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-27T07:47:51.762 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:47:52.006 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempita. 2024-04-27T07:47:52.032 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-27T07:47:52.064 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:47:52.073 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-waitress. 2024-04-27T07:47:52.100 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-27T07:47:52.143 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:47:52.350 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-paste. 2024-04-27T07:47:52.377 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-27T07:47:52.767 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:47:53.134 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-tempita. 2024-04-27T07:47:53.162 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-27T07:47:53.201 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:47:53.487 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-27T07:47:53.512 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-paste. 2024-04-27T07:47:53.513 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-27T07:47:53.539 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-27T07:47:53.553 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:47:53.580 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:47:53.856 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-27T07:47:53.883 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-27T07:47:53.890 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-27T07:47:53.906 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-27T07:47:53.922 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:47:53.948 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:47:54.184 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-27T07:47:54.200 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-27T07:47:54.217 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webtest. 2024-04-27T07:47:54.234 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:47:54.244 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-27T07:47:54.284 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-27T07:47:54.537 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webtest. 2024-04-27T07:47:54.564 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-27T07:47:54.595 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pecan. 2024-04-27T07:47:54.603 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-27T07:47:54.621 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-27T07:47:54.661 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:47:54.898 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pecan. 2024-04-27T07:47:54.925 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-27T07:47:54.956 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:47:55.024 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-certifi. 2024-04-27T07:47:55.051 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-27T07:47:55.091 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-27T07:47:55.286 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-certifi. 2024-04-27T07:47:55.302 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-27T07:47:55.335 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-27T07:47:55.428 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-idna. 2024-04-27T07:47:55.454 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-27T07:47:55.510 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-27T07:47:55.647 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-idna. 2024-04-27T07:47:55.674 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-27T07:47:55.713 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-27T07:47:55.813 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-urllib3. 2024-04-27T07:47:55.840 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-27T07:47:55.880 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:47:56.024 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-urllib3. 2024-04-27T07:47:56.051 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-27T07:47:56.091 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:47:56.208 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests. 2024-04-27T07:47:56.235 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-27T07:47:56.283 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:47:56.428 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-requests. 2024-04-27T07:47:56.455 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-27T07:47:56.517 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:47:56.569 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-27T07:47:56.596 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-27T07:47:56.627 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:47:56.830 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-27T07:47:56.857 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-27T07:47:56.896 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:47:56.922 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-27T07:47:56.949 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:56.988 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:57.183 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-27T07:47:57.210 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:47:57.249 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:57.368 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-27T07:47:57.396 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:57.435 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:57.762 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-27T07:47:57.790 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:57.829 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:57.954 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-27T07:47:57.981 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-27T07:47:58.012 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:47:58.140 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-27T07:47:58.168 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-27T07:47:58.206 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:47:58.315 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-openssl. 2024-04-27T07:47:58.342 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-27T07:47:58.382 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-27T07:47:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-openssl. 2024-04-27T07:47:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-27T07:47:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-27T07:47:58.710 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr. 2024-04-27T07:47:58.736 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:58.776 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:58.862 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr. 2024-04-27T07:47:58.890 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:58.920 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:59.138 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mon. 2024-04-27T07:47:59.164 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:59.204 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:59.290 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mon. 2024-04-27T07:47:59.317 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:59.356 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:47:59.783 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-osd. 2024-04-27T07:47:59.809 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:47:59.849 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:00.043 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-osd. 2024-04-27T07:48:00.070 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:00.101 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:00.911 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph. 2024-04-27T07:48:00.938 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../068-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:00.977 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:01.279 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph. 2024-04-27T07:48:01.296 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../068-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:01.297 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-fuse. 2024-04-27T07:48:01.326 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:01.337 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:01.363 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:01.573 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-fuse. 2024-04-27T07:48:01.590 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:01.623 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:01.700 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mds. 2024-04-27T07:48:01.727 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:01.757 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mds. 2024-04-27T07:48:01.987 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:02.026 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:02.177 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephadm. 2024-04-27T07:48:02.204 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../071-cephadm_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:02.243 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:02.420 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephadm. 2024-04-27T07:48:02.448 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../071-cephadm_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:02.487 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:02.571 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-27T07:48:02.598 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-27T07:48:02.638 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-27T07:48:02.814 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-27T07:48:02.842 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-27T07:48:02.882 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-27T07:48:02.940 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-27T07:48:02.956 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:48:02.998 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:03.217 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-27T07:48:03.244 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:48:03.283 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:04.089 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-babel. 2024-04-27T07:48:04.116 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:48:04.155 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:04.246 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-babel. 2024-04-27T07:48:04.263 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:48:04.304 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:04.475 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jinja2. 2024-04-27T07:48:04.502 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-27T07:48:04.541 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:48:04.624 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jinja2. 2024-04-27T07:48:04.651 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-27T07:48:04.711 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:48:04.894 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-27T07:48:04.922 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:04.961 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:05.052 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-27T07:48:05.080 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:05.119 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:05.314 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-27T07:48:05.341 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-27T07:48:05.380 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-27T07:48:05.472 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-27T07:48:05.499 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-27T07:48:05.538 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-27T07:48:05.666 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-routes. 2024-04-27T07:48:05.694 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-27T07:48:05.733 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:48:05.841 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-routes. 2024-04-27T07:48:05.868 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-27T07:48:05.907 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:48:06.086 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-27T07:48:06.114 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:06.153 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:06.244 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-27T07:48:06.271 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:06.310 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:07.366 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-27T07:48:07.381 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-27T07:48:07.415 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:48:07.473 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-27T07:48:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-27T07:48:07.523 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:48:07.945 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-joblib. 2024-04-27T07:48:07.972 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-27T07:48:08.120 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:48:08.177 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-joblib. 2024-04-27T07:48:08.205 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-27T07:48:08.243 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:48:08.556 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-27T07:48:08.583 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-27T07:48:08.605 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-27T07:48:08.633 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-27T07:48:08.708 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:48:08.757 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:48:09.019 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn. 2024-04-27T07:48:09.043 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn. 2024-04-27T07:48:09.046 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-27T07:48:09.070 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-27T07:48:09.085 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:48:09.109 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:48:09.768 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-27T07:48:09.797 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:09.834 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:09.878 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-27T07:48:09.905 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:09.944 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:10.704 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cachetools. 2024-04-27T07:48:10.732 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-27T07:48:10.771 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-27T07:48:10.897 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cachetools. 2024-04-27T07:48:10.924 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-27T07:48:10.964 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-27T07:48:11.024 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-27T07:48:11.051 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-27T07:48:11.090 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:48:11.225 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-27T07:48:11.252 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-27T07:48:11.283 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:48:11.385 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-27T07:48:11.401 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-27T07:48:11.443 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:48:11.612 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-27T07:48:11.638 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-27T07:48:11.678 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:48:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rsa. 2024-04-27T07:48:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-27T07:48:11.763 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-27T07:48:12.023 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rsa. 2024-04-27T07:48:12.050 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-27T07:48:12.074 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-google-auth. 2024-04-27T07:48:12.089 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-27T07:48:12.103 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-27T07:48:12.141 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-27T07:48:12.418 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-google-auth. 2024-04-27T07:48:12.436 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-27T07:48:12.445 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-27T07:48:12.464 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-27T07:48:12.484 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-27T07:48:12.502 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:48:12.783 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-27T07:48:12.795 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-27T07:48:12.814 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-websocket. 2024-04-27T07:48:12.829 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:48:12.842 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-27T07:48:12.912 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-27T07:48:13.249 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-websocket. 2024-04-27T07:48:13.250 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-27T07:48:13.276 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-27T07:48:13.278 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-27T07:48:13.315 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-27T07:48:13.316 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:48:13.668 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-27T07:48:13.696 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-27T07:48:13.743 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:48:13.955 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-27T07:48:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:14.005 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:14.300 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-27T07:48:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-27T07:48:14.366 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:48:14.399 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-27T07:48:14.426 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:14.465 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:14.696 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-27T07:48:14.724 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:48:14.734 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-27T07:48:14.762 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-27T07:48:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:48:14.792 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:48:14.990 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package jq. 2024-04-27T07:48:15.006 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:48:15.040 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-27T07:48:15.146 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-27T07:48:15.173 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:48:15.212 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:48:15.293 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package socat. 2024-04-27T07:48:15.309 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-27T07:48:15.351 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:48:15.473 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package jq. 2024-04-27T07:48:15.501 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:48:15.540 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-27T07:48:15.679 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package xmlstarlet. 2024-04-27T07:48:15.707 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-27T07:48:15.746 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:48:15.792 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package socat. 2024-04-27T07:48:15.820 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-27T07:48:15.858 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:48:16.092 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-test. 2024-04-27T07:48:16.108 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:16.150 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:16.170 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package xmlstarlet. 2024-04-27T07:48:16.186 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-27T07:48:16.220 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:48:16.566 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-test. 2024-04-27T07:48:16.582 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:16.624 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:18.763 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-volume. 2024-04-27T07:48:18.791 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:18.829 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:19.037 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-volume. 2024-04-27T07:48:19.064 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:19.095 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:19.208 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-27T07:48:19.237 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:19.275 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:19.432 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-27T07:48:19.459 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:19.490 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:19.586 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-colorama. 2024-04-27T07:48:19.614 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-27T07:48:19.653 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-27T07:48:19.776 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-colorama. 2024-04-27T07:48:19.804 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-27T07:48:19.843 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-27T07:48:19.905 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-shell. 2024-04-27T07:48:19.922 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:19.963 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.121 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephfs-shell. 2024-04-27T07:48:20.148 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:20.175 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-top. 2024-04-27T07:48:20.187 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.192 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:20.232 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.456 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephfs-top. 2024-04-27T07:48:20.472 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:48:20.485 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-27T07:48:20.513 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:20.514 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.552 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.783 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-27T07:48:20.811 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:20.850 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:20.871 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-27T07:48:20.899 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-27T07:48:20.938 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:48:21.144 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-27T07:48:21.172 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-27T07:48:21.202 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:48:21.293 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-27T07:48:21.321 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-27T07:48:21.359 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:48:21.540 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-27T07:48:21.568 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-27T07:48:21.607 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:48:21.697 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package nvme-cli. 2024-04-27T07:48:21.725 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-27T07:48:21.763 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:48:21.995 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package nvme-cli. 2024-04-27T07:48:22.021 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-27T07:48:22.061 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:48:22.168 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package pkg-config. 2024-04-27T07:48:22.197 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-27T07:48:22.234 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:48:22.457 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package pkg-config. 2024-04-27T07:48:22.485 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-27T07:48:22.496 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-27T07:48:22.523 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:48:22.524 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-27T07:48:22.563 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:48:22.751 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-27T07:48:22.767 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-27T07:48:22.801 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:48:22.999 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-27T07:48:23.028 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-27T07:48:23.066 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-27T07:48:23.204 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-27T07:48:23.232 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-27T07:48:23.262 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-27T07:48:23.318 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastescript. 2024-04-27T07:48:23.346 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-27T07:48:23.385 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-27T07:48:23.540 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastescript. 2024-04-27T07:48:23.568 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-27T07:48:23.606 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-27T07:48:23.714 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pluggy. 2024-04-27T07:48:23.742 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-27T07:48:23.780 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:48:23.927 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pluggy. 2024-04-27T07:48:23.954 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-27T07:48:23.993 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:48:24.092 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-psutil. 2024-04-27T07:48:24.120 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-27T07:48:24.158 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-27T07:48:24.330 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-psutil. 2024-04-27T07:48:24.357 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-27T07:48:24.396 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-27T07:48:24.486 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-py. 2024-04-27T07:48:24.514 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-27T07:48:24.552 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-27T07:48:24.732 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-py. 2024-04-27T07:48:24.760 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-27T07:48:24.798 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-27T07:48:24.881 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pygments. 2024-04-27T07:48:24.910 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-27T07:48:24.948 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:48:25.127 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pygments. 2024-04-27T07:48:25.154 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-27T07:48:25.193 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:48:25.418 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-27T07:48:25.447 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-27T07:48:25.485 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:48:25.622 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-27T07:48:25.650 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-27T07:48:25.680 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:48:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-toml. 2024-04-27T07:48:25.824 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-27T07:48:25.862 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-27T07:48:25.983 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-toml. 2024-04-27T07:48:26.011 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-27T07:48:26.049 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-27T07:48:26.140 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pytest. 2024-04-27T07:48:26.168 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-27T07:48:26.206 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:48:26.361 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pytest. 2024-04-27T07:48:26.389 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-27T07:48:26.427 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:48:26.544 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplejson. 2024-04-27T07:48:26.572 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-27T07:48:26.610 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:48:26.773 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplejson. 2024-04-27T07:48:26.800 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-27T07:48:26.839 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:48:26.888 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-27T07:48:26.913 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-27T07:48:26.963 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:48:27.108 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-27T07:48:27.136 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-27T07:48:27.175 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:48:27.650 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package radosgw. 2024-04-27T07:48:27.678 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../122-radosgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:27.716 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:27.929 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package radosgw. 2024-04-27T07:48:27.957 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../122-radosgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:27.995 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:28.545 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package rbd-fuse. 2024-04-27T07:48:28.562 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:28.595 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:28.774 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-fuse. 2024-04-27T07:48:28.790 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:48:28.823 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:29.143 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-27T07:48:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-27T07:48:29.517 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:48:29.611 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:48:29.643 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:48:29.737 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:48:29.909 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-27T07:48:29.997 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-27T07:48:30.245 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-27T07:48:30.312 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-27T07:48:30.688 INFO:teuthology.orchestra.run.smithi022.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-27T07:48:30.710 INFO:teuthology.orchestra.run.smithi183.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-27T07:48:30.752 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:30.772 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:31.227 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user cephadm....done 2024-04-27T07:48:31.289 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user cephadm....done 2024-04-27T07:48:31.376 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:48:31.453 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:48:31.660 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:48:31.729 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:48:31.903 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:48:31.981 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:48:32.038 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:48:32.090 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:48:32.281 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-27T07:48:32.358 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-27T07:48:32.540 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:48:32.625 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:48:32.649 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-27T07:48:32.752 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-27T07:48:32.959 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:48:33.062 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:48:33.328 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-27T07:48:33.430 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-27T07:48:33.612 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-27T07:48:33.664 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-27T07:48:33.872 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-27T07:48:33.907 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-27T07:48:34.148 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:48:34.158 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:48:34.417 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:48:34.427 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:48:34.668 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:48:34.678 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:48:34.927 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:48:34.946 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:48:35.195 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:35.255 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:48:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:48:35.607 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:48:35.723 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-27T07:48:35.733 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-27T07:48:35.982 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:48:35.992 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:48:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:48:36.127 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:48:36.368 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:48:36.386 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:48:36.709 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-27T07:48:36.741 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-27T07:48:36.805 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:48:36.834 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-27T07:48:36.867 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-27T07:48:36.906 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-27T07:48:36.914 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:48:37.015 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-27T07:48:37.266 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:37.358 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:37.391 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-27T07:48:37.484 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-27T07:48:37.693 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:48:37.752 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:48:37.952 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:48:38.012 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:48:38.237 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up zip (3.0-12build2) ... 2024-04-27T07:48:38.313 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up zip (3.0-12build2) ... 2024-04-27T07:48:38.363 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:48:38.422 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:48:39.082 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:48:39.132 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:48:39.383 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:48:39.491 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:48:39.509 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:48:39.609 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:48:39.644 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:48:39.710 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:48:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-27T07:48:40.011 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-27T07:48:40.355 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-27T07:48:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-27T07:48:40.581 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:48:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:48:40.932 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:48:41.048 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:48:41.217 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-27T07:48:41.341 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-27T07:48:41.534 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:48:41.659 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:48:41.886 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:48:42.044 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:48:42.145 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:48:42.271 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:42.312 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:48:42.446 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:42.564 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-idna (3.3-1) ... 2024-04-27T07:48:42.748 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-idna (3.3-1) ... 2024-04-27T07:48:42.874 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:48:43.082 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:48:44.038 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:48:44.256 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:44.297 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:48:44.390 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-27T07:48:44.548 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:44.682 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-27T07:48:44.691 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:48:44.818 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:48:44.950 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:48:44.952 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:48:45.076 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:48:45.202 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:48:45.253 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:48:45.504 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-27T07:48:45.521 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:48:45.730 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:45.780 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-27T07:48:45.856 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:48:46.040 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:46.157 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:48:46.166 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:48:46.417 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-27T07:48:46.492 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:48:46.685 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-27T07:48:46.760 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-27T07:48:46.945 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:48:47.002 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-27T07:48:47.196 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-27T07:48:47.245 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:48:47.463 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:48:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-27T07:48:47.564 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-27T07:48:47.756 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:48:47.832 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:48:47.882 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-27T07:48:47.950 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:48:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:48:48.242 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:48:48.301 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:48:48.537 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:48:48.611 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:48:48.772 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:48:48.898 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:48:48.920 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:48:49.023 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:49.112 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-27T07:48:49.163 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:48:49.264 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:48:49.350 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:48:49.382 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:48:49.471 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-27T07:48:49.476 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:48:49.725 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:48:49.843 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:48:50.003 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:48:50.236 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:48:50.270 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-27T07:48:50.388 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-27T07:48:50.496 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-27T07:48:50.622 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-27T07:48:50.622 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:50.748 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:48:50.881 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:48:51.007 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:48:51.057 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:48:51.317 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:48:51.350 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:48:51.635 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-27T07:48:51.660 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:48:51.855 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-27T07:48:51.927 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-27T07:48:52.097 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-27T07:48:52.139 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:48:52.407 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-27T07:48:52.599 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:48:52.691 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:48:52.951 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:48:52.976 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-27T07:48:53.261 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:48:53.353 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-27T07:48:53.621 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:48:53.639 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:48:53.739 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:53.841 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:53.951 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:48:53.966 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:48:54.092 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.218 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.285 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.344 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:48:54.411 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.537 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-27T07:48:54.671 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.772 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.813 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:54.882 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-27T07:48:54.956 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.082 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.183 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.284 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.292 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:48:55.292 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:48:55.385 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.595 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:48:55.595 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:48:55.783 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:55.909 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.044 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.095 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.196 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.230 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.355 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.481 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:56.580 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:48:56.607 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:48:57.886 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:57.925 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.037 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.051 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.163 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.177 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.289 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.303 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.412 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.513 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:48:59.018 INFO:teuthology.orchestra.run.smithi022.stdout:Adding group ceph....done 2024-04-27T07:48:59.039 INFO:teuthology.orchestra.run.smithi183.stdout:Adding group ceph....done 2024-04-27T07:48:59.394 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user ceph....done 2024-04-27T07:48:59.416 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user ceph....done 2024-04-27T07:48:59.526 INFO:teuthology.orchestra.run.smithi183.stdout:Setting system user ceph properties....done 2024-04-27T07:48:59.538 INFO:teuthology.orchestra.run.smithi183.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-27T07:48:59.563 INFO:teuthology.orchestra.run.smithi022.stdout:Setting system user ceph properties....done 2024-04-27T07:48:59.579 INFO:teuthology.orchestra.run.smithi022.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-27T07:48:59.675 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-27T07:48:59.722 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-27T07:48:59.974 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-27T07:49:00.067 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-27T07:49:00.439 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:00.539 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:00.743 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:49:00.743 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:49:00.755 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:49:00.756 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:49:01.173 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:01.240 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:01.299 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:01.366 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:01.752 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:49:01.753 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:49:01.815 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:49:01.815 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:49:02.209 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:02.293 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:02.546 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-27T07:49:02.630 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-27T07:49:02.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:03.136 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:03.209 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:49:03.209 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:49:03.376 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:49:03.376 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:49:03.688 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:03.882 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:03.900 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:49:03.900 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:49:04.101 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:49:04.101 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:49:04.363 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:04.641 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:04.648 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:49:04.648 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:49:04.908 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:49:04.908 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:49:05.200 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.325 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.443 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.476 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.552 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.688 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:49:05.688 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:49:05.703 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:05.929 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:49:05.929 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:49:06.169 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.295 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.396 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.422 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.530 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.581 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.707 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.858 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:49:06.875 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:49:06.984 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:49:07.136 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:49:07.244 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:49:07.354 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:49:07.489 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:49:08.295 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:49:08.649 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:49:08.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-27T07:49:09.005 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-04-27T07:49:09.185 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-04-27T07:49:09.186 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-04-27T07:49:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:49:09.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-27T07:49:09.346 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:49:09.355 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:49:09.356 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:49:09.356 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:49:09.356 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:49:09.357 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-04-27T07:49:09.357 INFO:teuthology.orchestra.run.smithi183.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-27T07:49:09.357 INFO:teuthology.orchestra.run.smithi183.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-27T07:49:09.357 INFO:teuthology.orchestra.run.smithi183.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-27T07:49:09.358 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-27T07:49:09.358 INFO:teuthology.orchestra.run.smithi183.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:49:09.359 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-04-27T07:49:09.359 INFO:teuthology.orchestra.run.smithi183.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-27T07:49:09.359 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-27T07:49:09.359 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-27T07:49:09.399 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-27T07:49:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:49:09.401 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2024-04-27T07:49:09.401 INFO:teuthology.orchestra.run.smithi183.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-27T07:49:09.486 INFO:teuthology.orchestra.run.smithi183.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-27T07:49:09.486 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 55.7 MB of archives. 2024-04-27T07:49:09.486 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-27T07:49:09.486 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-27T07:49:09.551 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-04-27T07:49:09.552 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-04-27T07:49:09.723 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-27T07:49:09.730 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-27T07:49:09.752 INFO:teuthology.orchestra.run.smithi022.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:49:09.753 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:49:09.754 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:49:09.754 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:49:09.754 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-04-27T07:49:09.755 INFO:teuthology.orchestra.run.smithi022.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-27T07:49:09.755 INFO:teuthology.orchestra.run.smithi022.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-27T07:49:09.755 INFO:teuthology.orchestra.run.smithi022.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-27T07:49:09.755 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-27T07:49:09.755 INFO:teuthology.orchestra.run.smithi022.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:49:09.757 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-04-27T07:49:09.757 INFO:teuthology.orchestra.run.smithi022.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-27T07:49:09.757 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-27T07:49:09.757 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-27T07:49:09.795 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-27T07:49:09.796 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-27T07:49:09.797 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-27T07:49:09.801 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-04-27T07:49:09.802 INFO:teuthology.orchestra.run.smithi022.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-27T07:49:09.802 INFO:teuthology.orchestra.run.smithi022.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-27T07:49:09.803 INFO:teuthology.orchestra.run.smithi022.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-27T07:49:09.803 INFO:teuthology.orchestra.run.smithi022.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-27T07:49:09.803 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-27T07:49:09.803 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-27T07:49:09.803 INFO:teuthology.orchestra.run.smithi022.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:49:09.804 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2024-04-27T07:49:09.805 INFO:teuthology.orchestra.run.smithi022.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-27T07:49:09.815 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-27T07:49:09.816 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-27T07:49:09.824 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-27T07:49:09.826 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-27T07:49:09.826 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-27T07:49:09.828 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-27T07:49:09.830 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-27T07:49:09.850 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-27T07:49:09.850 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-27T07:49:09.852 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-27T07:49:09.869 INFO:teuthology.orchestra.run.smithi022.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-27T07:49:09.869 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 55.7 MB of archives. 2024-04-27T07:49:09.869 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-27T07:49:09.869 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-27T07:49:10.177 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-27T07:49:10.185 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-27T07:49:10.242 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-27T07:49:10.244 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-27T07:49:10.245 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-27T07:49:10.248 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-27T07:49:10.248 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-27T07:49:10.253 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-27T07:49:10.254 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-27T07:49:10.255 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-27T07:49:10.256 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-27T07:49:10.256 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-27T07:49:10.266 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-27T07:49:10.266 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-27T07:49:10.274 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-27T07:49:10.275 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-27T07:49:10.279 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-27T07:49:10.281 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-27T07:49:10.282 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-27T07:49:10.284 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-27T07:49:10.286 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-27T07:49:10.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-27T07:49:10.293 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-27T07:49:10.305 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-27T07:49:10.308 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-27T07:49:10.309 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-27T07:49:10.311 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-27T07:49:10.344 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-27T07:49:10.348 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-27T07:49:10.348 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-27T07:49:10.349 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-27T07:49:10.349 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-27T07:49:10.366 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-27T07:49:10.370 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-27T07:49:10.801 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-27T07:49:10.802 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-27T07:49:10.802 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-27T07:49:10.802 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-27T07:49:10.802 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-27T07:49:10.802 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-27T07:49:10.823 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-27T07:49:10.823 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-27T07:49:10.909 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-27T07:49:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-27T07:49:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-27T07:49:10.918 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-27T07:49:10.918 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-27T07:49:10.925 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-27T07:49:10.927 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-27T07:49:10.938 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-27T07:49:10.938 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-27T07:49:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-27T07:49:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-27T07:49:10.983 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-27T07:49:11.006 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-27T07:49:11.012 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-27T07:49:11.012 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-27T07:49:11.013 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-27T07:49:11.014 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-27T07:49:11.032 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-27T07:49:11.034 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-27T07:49:11.228 INFO:teuthology.orchestra.run.smithi183.stdout:Preconfiguring packages ... 2024-04-27T07:49:11.274 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-27T07:49:11.274 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-27T07:49:11.275 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-27T07:49:11.275 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-27T07:49:11.310 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-27T07:49:11.310 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-27T07:49:11.323 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-27T07:49:11.323 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-27T07:49:11.381 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 55.7 MB in 1s (39.4 MB/s) 2024-04-27T07:49:11.707 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:49:11.712 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:Preconfiguring packages ... 2024-04-27T07:49:11.860 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:11.900 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 55.7 MB in 2s (36.9 MB/s) 2024-04-27T07:49:12.161 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:49:12.167 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:12.296 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:12.463 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:12.903 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:13.391 INFO:teuthology.orchestra.run.smithi183.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-27T07:49:13.655 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package flex. 2024-04-27T07:49:13.752 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:49:13.758 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:13.794 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-27T07:49:13.860 INFO:teuthology.orchestra.run.smithi022.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-27T07:49:14.112 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package flex. 2024-04-27T07:49:14.218 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:49:14.249 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:14.268 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-27T07:49:14.318 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:49:14.491 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-27T07:49:14.752 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:49:14.911 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:14.924 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-27T07:49:15.327 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:15.594 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:49:15.599 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:15.741 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:15.851 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:49:15.856 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:15.982 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:16.208 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:16.339 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:49:16.514 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:49:16.675 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:16.816 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:17.065 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package bison. 2024-04-27T07:49:17.170 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:49:17.176 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-27T07:49:17.198 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package bison. 2024-04-27T07:49:17.225 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:49:17.305 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:49:17.311 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-27T07:49:17.367 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:49:17.726 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-27T07:49:17.753 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-27T07:49:17.768 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-27T07:49:17.796 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-27T07:49:17.801 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:49:17.834 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:49:18.138 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package dump. 2024-04-27T07:49:18.154 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-27T07:49:18.196 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking dump (0.4b47-3) ... 2024-04-27T07:49:18.254 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package dump. 2024-04-27T07:49:18.281 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-27T07:49:18.329 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking dump (0.4b47-3) ... 2024-04-27T07:49:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package indent. 2024-04-27T07:49:18.528 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:18.566 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:49:18.624 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package indent. 2024-04-27T07:49:18.651 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:18.703 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:49:18.828 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-27T07:49:18.845 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-27T07:49:18.886 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:49:18.993 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-27T07:49:19.009 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-27T07:49:19.059 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:49:19.190 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-27T07:49:19.219 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-27T07:49:19.248 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:49:19.422 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-27T07:49:19.450 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-27T07:49:19.488 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:49:19.493 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-27T07:49:19.510 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-27T07:49:19.551 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:49:19.733 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-27T07:49:19.760 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-27T07:49:19.799 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:49:19.888 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-27T07:49:19.917 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:19.955 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:20.194 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-27T07:49:20.208 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-27T07:49:20.222 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:20.236 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:20.260 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:20.274 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:20.502 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjson-perl. 2024-04-27T07:49:20.519 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-27T07:49:20.560 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-27T07:49:20.756 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-27T07:49:20.784 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-27T07:49:20.872 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-27T07:49:20.900 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-27T07:49:20.938 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:49:21.039 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:21.291 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjson-perl. 2024-04-27T07:49:21.307 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-27T07:49:21.349 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-27T07:49:21.703 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-27T07:49:21.731 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-27T07:49:21.786 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:49:22.683 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-27T07:49:22.712 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:49:22.750 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:22.977 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-27T07:49:22.994 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:49:23.036 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:23.355 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-27T07:49:23.384 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:49:23.397 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-27T07:49:23.413 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:49:23.421 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:23.447 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:23.699 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-27T07:49:23.727 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:49:23.766 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:23.775 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-27T07:49:23.803 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-27T07:49:23.841 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:49:24.110 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-27T07:49:24.136 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-27T07:49:24.138 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:49:24.165 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:49:24.176 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:24.194 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:49:24.505 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-27T07:49:24.533 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-27T07:49:24.539 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-27T07:49:24.568 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:24.571 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:49:24.605 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:24.841 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-27T07:49:24.869 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:49:24.907 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:49:24.951 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-27T07:49:24.979 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:25.017 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:25.260 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-27T07:49:25.288 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:25.311 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-27T07:49:25.326 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:25.328 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:25.370 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-27T07:49:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:49:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:25.705 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-27T07:49:25.733 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:25.771 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:26.074 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:26.175 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:26.184 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-27T07:49:26.213 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-27T07:49:26.251 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:49:26.469 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-27T07:49:26.496 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:49:26.544 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:26.788 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package modemmanager. 2024-04-27T07:49:26.816 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:49:26.854 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:27.048 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-27T07:49:27.075 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-27T07:49:27.114 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:49:27.467 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package modemmanager. 2024-04-27T07:49:27.478 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager. 2024-04-27T07:49:27.495 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:49:27.507 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:49:27.533 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:27.553 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:49:28.165 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package network-manager. 2024-04-27T07:49:28.169 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ppp. 2024-04-27T07:49:28.193 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:49:28.198 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-27T07:49:28.239 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:49:28.460 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:49:28.757 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package pptp-linux. 2024-04-27T07:49:28.786 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-27T07:49:28.824 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-27T07:49:28.864 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ppp. 2024-04-27T07:49:28.891 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-27T07:49:29.026 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-27T07:49:29.043 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-27T07:49:29.085 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-27T07:49:29.176 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:49:29.388 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-27T07:49:29.405 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-27T07:49:29.446 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-common (238) ... 2024-04-27T07:49:29.501 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package pptp-linux. 2024-04-27T07:49:29.529 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-27T07:49:29.568 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-27T07:49:29.759 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-27T07:49:29.787 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:29.825 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:29.829 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-27T07:49:29.845 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-27T07:49:30.123 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-27T07:49:30.139 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-27T07:49:30.173 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-client-common (238) ... 2024-04-27T07:49:30.372 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-common. 2024-04-27T07:49:30.400 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-27T07:49:30.449 INFO:teuthology.orchestra.run.smithi022.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-27T07:49:30.477 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-27T07:49:30.478 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-common (238) ... 2024-04-27T07:49:30.505 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:30.543 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:30.954 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-14. 2024-04-27T07:49:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:31.024 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:31.148 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-common. 2024-04-27T07:49:31.176 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-27T07:49:31.241 INFO:teuthology.orchestra.run.smithi183.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-27T07:49:31.271 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-common (238) ... 2024-04-27T07:49:31.715 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-14. 2024-04-27T07:49:31.743 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:49:31.785 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:32.216 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql. 2024-04-27T07:49:32.234 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-27T07:49:32.274 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql (14+238) ... 2024-04-27T07:49:32.552 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client. 2024-04-27T07:49:32.580 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-27T07:49:32.966 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client (14+238) ... 2024-04-27T07:49:33.212 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-27T07:49:33.230 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-27T07:49:33.270 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-27T07:49:33.285 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql. 2024-04-27T07:49:33.302 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-27T07:49:33.351 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql (14+238) ... 2024-04-27T07:49:33.548 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package tcl8.6. 2024-04-27T07:49:33.577 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-27T07:49:33.587 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-client. 2024-04-27T07:49:33.606 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:49:33.616 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-27T07:49:33.654 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-client (14+238) ... 2024-04-27T07:49:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package tcl. 2024-04-27T07:49:33.879 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-27T07:49:33.923 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-27T07:49:33.931 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-27T07:49:33.960 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-27T07:49:33.998 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-27T07:49:34.271 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package wpasupplicant. 2024-04-27T07:49:34.284 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package tcl8.6. 2024-04-27T07:49:34.299 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-27T07:49:34.312 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-27T07:49:34.337 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:49:34.350 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:49:34.603 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package tcl. 2024-04-27T07:49:34.633 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-27T07:49:34.675 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-27T07:49:34.700 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-27T07:49:34.718 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-27T07:49:34.750 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-27T07:49:35.031 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package wpasupplicant. 2024-04-27T07:49:35.059 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-27T07:49:35.098 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:49:35.154 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-27T07:49:35.182 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-27T07:49:35.228 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:49:35.451 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-27T07:49:35.467 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-27T07:49:35.509 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-27T07:49:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-common (238) ... 2024-04-27T07:49:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-27T07:49:35.821 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-27T07:49:35.837 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-27T07:49:35.860 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:49:35.879 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:49:35.986 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:49:36.184 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-client-common (238) ... 2024-04-27T07:49:36.205 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:49:36.331 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-27T07:49:36.519 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:49:36.629 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:36.730 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:49:36.856 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:49:36.884 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-27T07:49:36.979 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:49:36.982 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:37.364 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:49:37.364 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:49:37.465 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-27T07:49:37.554 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:49:37.873 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:37.928 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:49:37.928 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:49:37.999 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:49:38.135 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:38.260 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-27T07:49:38.386 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:49:38.390 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:49:38.487 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:38.491 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:49:38.588 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:49:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:49:38.743 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-27T07:49:38.869 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:49:38.995 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:39.121 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:49:39.247 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:49:39.378 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:49:39.487 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up dump (0.4b47-3) ... 2024-04-27T07:49:39.565 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-27T07:49:39.630 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:39.756 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:49:39.852 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:49:39.883 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-27T07:49:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up dump (0.4b47-3) ... 2024-04-27T07:49:40.009 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-27T07:49:40.072 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-27T07:49:40.135 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:49:40.138 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:40.225 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-27T07:49:40.295 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:49:40.381 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:49:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:49:40.507 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-27T07:49:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:49:40.633 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-27T07:49:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:40.751 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:49:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-27T07:49:40.832 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-27T07:49:40.894 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:49:40.969 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client (14+238) ... 2024-04-27T07:49:41.011 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:49:41.095 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:49:41.188 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:49:41.339 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:41.465 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout:which means they are currently configured by ifupdown: 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout:- ens1f0 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-27T07:49:41.562 INFO:teuthology.orchestra.run.smithi022.stdout:remove their configuration from /etc/network/interfaces. 2024-04-27T07:49:41.563 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:41.616 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-client (14+238) ... 2024-04-27T07:49:41.704 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-27T07:49:41.717 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:49:42.052 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-27T07:49:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:42.169 INFO:teuthology.orchestra.run.smithi183.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-27T07:49:42.169 INFO:teuthology.orchestra.run.smithi183.stdout:which means they are currently configured by ifupdown: 2024-04-27T07:49:42.170 INFO:teuthology.orchestra.run.smithi183.stdout:- enp3s0f1 2024-04-27T07:49:42.170 INFO:teuthology.orchestra.run.smithi183.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-27T07:49:42.170 INFO:teuthology.orchestra.run.smithi183.stdout:remove their configuration from /etc/network/interfaces. 2024-04-27T07:49:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:42.297 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-27T07:49:42.402 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-27T07:49:42.596 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-27T07:49:42.922 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:42.939 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-27T07:49:43.099 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-27T07:49:43.394 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:49:43.538 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:49:43.664 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-27T07:49:43.734 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-common (238) ... 2024-04-27T07:49:43.773 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-common (238) ... 2024-04-27T07:49:44.622 INFO:teuthology.orchestra.run.smithi022.stdout:Adding user postgres to group ssl-cert 2024-04-27T07:49:44.642 INFO:teuthology.orchestra.run.smithi183.stdout:Adding user postgres to group ssl-cert 2024-04-27T07:49:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:44.845 INFO:teuthology.orchestra.run.smithi183.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-27T07:49:44.854 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-27T07:49:45.016 INFO:teuthology.orchestra.run.smithi183.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-27T07:49:45.016 INFO:teuthology.orchestra.run.smithi183.stdout:Removing obsolete dictionary files: 2024-04-27T07:49:45.066 INFO:teuthology.orchestra.run.smithi022.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-27T07:49:45.067 INFO:teuthology.orchestra.run.smithi022.stdout:Removing obsolete dictionary files: 2024-04-27T07:49:45.511 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-27T07:49:45.668 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-27T07:49:45.875 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:46.019 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:49:46.186 INFO:teuthology.orchestra.run.smithi183.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-27T07:49:46.203 INFO:teuthology.orchestra.run.smithi183.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:This user must also own the server process. 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:The database cluster will be initialized with locale "en_US". 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:The default text search configuration will be set to "english". 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:Data page checksums are disabled. 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-27T07:49:46.251 INFO:teuthology.orchestra.run.smithi183.stdout:creating subdirectories ... ok 2024-04-27T07:49:46.251 INFO:teuthology.orchestra.run.smithi183.stdout:selecting dynamic shared memory implementation ... posix 2024-04-27T07:49:46.266 INFO:teuthology.orchestra.run.smithi183.stdout:selecting default max_connections ... 100 2024-04-27T07:49:46.285 INFO:teuthology.orchestra.run.smithi183.stdout:selecting default shared_buffers ... 128MB 2024-04-27T07:49:46.299 INFO:teuthology.orchestra.run.smithi183.stdout:selecting default time zone ... Etc/UTC 2024-04-27T07:49:46.300 INFO:teuthology.orchestra.run.smithi183.stdout:creating configuration files ... ok 2024-04-27T07:49:46.340 INFO:teuthology.orchestra.run.smithi022.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-27T07:49:46.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-27T07:49:46.408 INFO:teuthology.orchestra.run.smithi022.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-27T07:49:46.408 INFO:teuthology.orchestra.run.smithi022.stdout:This user must also own the server process. 2024-04-27T07:49:46.408 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:46.408 INFO:teuthology.orchestra.run.smithi022.stdout:The database cluster will be initialized with locale "en_US". 2024-04-27T07:49:46.408 INFO:teuthology.orchestra.run.smithi022.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:The default text search configuration will be set to "english". 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:Data page checksums are disabled. 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:creating subdirectories ... ok 2024-04-27T07:49:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:selecting dynamic shared memory implementation ... posix 2024-04-27T07:49:46.417 INFO:teuthology.orchestra.run.smithi183.stdout:running bootstrap script ... ok 2024-04-27T07:49:46.426 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default max_connections ... 100 2024-04-27T07:49:46.446 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default shared_buffers ... 128MB 2024-04-27T07:49:46.453 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default time zone ... Etc/UTC 2024-04-27T07:49:46.453 INFO:teuthology.orchestra.run.smithi022.stdout:creating configuration files ... ok 2024-04-27T07:49:46.573 INFO:teuthology.orchestra.run.smithi022.stdout:running bootstrap script ... ok 2024-04-27T07:49:47.028 INFO:teuthology.orchestra.run.smithi183.stdout:performing post-bootstrap initialization ... ok 2024-04-27T07:49:47.199 INFO:teuthology.orchestra.run.smithi022.stdout:performing post-bootstrap initialization ... ok 2024-04-27T07:49:47.804 INFO:teuthology.orchestra.run.smithi022.stdout:syncing data to disk ... ok 2024-04-27T07:49:48.003 INFO:teuthology.orchestra.run.smithi183.stdout:syncing data to disk ... ok 2024-04-27T07:49:48.222 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-27T07:49:48.398 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-27T07:49:50.975 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:51.101 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-27T07:49:51.123 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:51.232 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-27T07:49:51.252 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql (14+238) ... 2024-04-27T07:49:51.358 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up postgresql (14+238) ... 2024-04-27T07:49:51.378 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:51.484 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:49:51.503 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:51.611 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:49:51.756 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:49:51.757 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:49:51.864 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:49:51.864 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:49:52.340 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:49:52.363 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:49:52.432 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-27T07:49:52.472 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-27T07:49:52.734 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:49:52.766 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:49:52.878 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:49:52.909 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:49:54.181 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-27T07:49:54.229 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-27T07:49:54.289 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:49:54.337 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:49:54.778 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:49:54.784 DEBUG:teuthology.parallel:result is None 2024-04-27T07:49:55.524 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:49:55.528 DEBUG:teuthology.parallel:result is None 2024-04-27T07:49:55.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:49:55.668 DEBUG:teuthology.orchestra.run.smithi022:> dpkg-query -W -f '${Version}' ceph 2024-04-27T07:49:55.688 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:49:55.689 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:49:55.689 INFO:teuthology.task.install:The correct ceph version 19.0.0-3227-gc37f63b3-1jammy is installed. 2024-04-27T07:49:55.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:49:55.829 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2024-04-27T07:49:55.848 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:49:55.849 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:49:55.849 INFO:teuthology.task.install:The correct ceph version 19.0.0-3227-gc37f63b3-1jammy is installed. 2024-04-27T07:49:55.851 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T07:49:55.851 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:55.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T07:49:55.866 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:55.866 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T07:49:55.904 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T07:49:55.904 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:55.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T07:49:55.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T07:49:55.978 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:55.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T07:49:55.993 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T07:49:56.048 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T07:49:56.049 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:56.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T07:49:56.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T07:49:56.119 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:56.120 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T07:49:56.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T07:49:56.188 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T07:49:56.188 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:56.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T07:49:56.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T07:49:56.255 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:56.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T07:49:56.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T07:49:56.323 INFO:teuthology.run_tasks:Running task ceph... 2024-04-27T07:49:56.411 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-27T07:49:56.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2024-04-27T07:49:56.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2024-04-27T07:49:56.426 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-27T07:49:56.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T07:49:56.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T07:49:56.479 INFO:tasks.ceph:Creating extra log directories... 2024-04-27T07:49:56.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T07:49:56.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T07:49:56.543 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-27T07:49:56.543 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-27T07:49:56.543 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-27T07:49:56.543 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:56.543 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-27T07:49:56.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:56.660 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:56.660 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-27T07:49:56.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:56.745 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-27T07:49:56.745 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-27T07:49:56.746 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672096', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4710, 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'sleep_before_teardown': 0, 'subset': '356/512', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi022.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBK5//N9I5E8X52d9w0riAGxZo/yEWmWVfz5IcTd0tHDDmpIi0mfkrfZpCBvNn7ogMpSAenSCGWIcXBnCuYlgvc=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII9zDJkhPI0/3gDcjcctfIsRaLoXjYWnEsO4xBm3NFpKrd5ss4yUnjoi6O8bJpJ1kOqbgWwtLoG3PIc9Eesj4k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_20:24:13', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-27T07:49:56.746 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T07:49:56.749 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T07:49:56.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T07:49:56.795 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T07:49:56.844 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:56.845 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T07:49:56.855 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T07:49:56.855 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-27 07:43:47.420601498 +0000 2024-04-27T07:49:56.903 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-27 07:43:47.172603713 +0000 2024-04-27T07:49:56.904 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-27 07:43:47.172603713 +0000 2024-04-27T07:49:56.904 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-04-27T07:49:56.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T07:49:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-27T07:49:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-27T07:49:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000312275 s, 1.6 MB/s 2024-04-27T07:49:56.959 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T07:49:57.008 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-27 07:43:47.652599426 +0000 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-27 07:43:47.648599462 +0000 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-27 07:43:47.648599462 +0000 2024-04-27T07:49:57.055 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-04-27T07:49:57.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T07:49:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-27T07:49:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-27T07:49:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000314021 s, 1.6 MB/s 2024-04-27T07:49:57.115 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T07:49:57.163 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-27 07:43:48.460592209 +0000 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-27 07:43:48.192594602 +0000 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-27 07:43:48.192594602 +0000 2024-04-27T07:49:57.211 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-04-27T07:49:57.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T07:49:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-27T07:49:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-27T07:49:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000300007 s, 1.7 MB/s 2024-04-27T07:49:57.266 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T07:49:57.316 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-04-27T07:49:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T07:49:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.366 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-04-27T07:49:57.366 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.367 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-27 07:43:48.916588136 +0000 2024-04-27T07:49:57.367 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-27 07:43:48.672590315 +0000 2024-04-27T07:49:57.367 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-27 07:43:48.672590315 +0000 2024-04-27T07:49:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-04-27T07:49:57.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T07:49:57.421 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-27T07:49:57.421 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-27T07:49:57.421 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.00029323 s, 1.7 MB/s 2024-04-27T07:49:57.423 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T07:49:57.476 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:49:57.476 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:57.476 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T07:49:57.482 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T07:49:57.482 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-27 07:43:44.486720694 +0000 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-27 07:43:44.234722945 +0000 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-27 07:43:44.234722945 +0000 2024-04-27T07:49:57.529 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-04-27T07:49:57.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T07:49:57.583 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-27T07:49:57.583 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-27T07:49:57.583 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00042664 s, 1.2 MB/s 2024-04-27T07:49:57.584 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T07:49:57.634 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-27 07:43:44.734718478 +0000 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-27 07:43:44.730718514 +0000 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-27 07:43:44.730718514 +0000 2024-04-27T07:49:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-04-27T07:49:57.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T07:49:57.735 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-27T07:49:57.735 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-27T07:49:57.735 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000424206 s, 1.2 MB/s 2024-04-27T07:49:57.736 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T07:49:57.786 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-27 07:43:45.458712011 +0000 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-27 07:43:45.194714369 +0000 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-27 07:43:45.194714369 +0000 2024-04-27T07:49:57.833 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-04-27T07:49:57.834 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T07:49:57.886 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-27T07:49:57.887 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-27T07:49:57.887 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000321412 s, 1.6 MB/s 2024-04-27T07:49:57.888 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T07:49:57.938 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-27 07:43:45.990707260 +0000 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-27 07:43:45.718709688 +0000 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-27 07:43:45.718709688 +0000 2024-04-27T07:49:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-04-27T07:49:57.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T07:49:58.039 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-27T07:49:58.039 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-27T07:49:58.039 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000377147 s, 1.4 MB/s 2024-04-27T07:49:58.041 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T07:49:58.089 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:49:58.090 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-27T07:49:58.090 INFO:tasks.ceph:Generating config... 2024-04-27T07:49:58.091 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-27T07:49:58.091 INFO:tasks.ceph:[client] debug client = 20 2024-04-27T07:49:58.091 INFO:tasks.ceph:[client] debug ms = 1 2024-04-27T07:49:58.091 INFO:tasks.ceph:[client] pid file = 2024-04-27T07:49:58.092 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-27T07:49:58.092 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-27T07:49:58.092 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-27T07:49:58.093 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-27T07:49:58.093 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-27T07:49:58.093 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-27T07:49:58.093 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-27T07:49:58.093 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-27T07:49:58.093 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-27T07:49:58.094 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-27T07:49:58.094 INFO:tasks.ceph:Setting up mon.a... 2024-04-27T07:49:58.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-27T07:49:58.128 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2024-04-27T07:49:58.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-27T07:49:58.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:49:58.270 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22'), ('mon.b', '172.21.15.183'), ('mon.c', '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]')] 2024-04-27T07:49:58.271 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-27T07:49:58.272 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:58.272 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T07:49:58.320 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --add b 172.21.15.183 --addv c '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid 807b2bc3-0b23-4239-8346-8619bcda9f4e 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 807b2bc3-0b23-4239-8346-8619bcda9f4e 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2024-04-27T07:49:58.411370+0000 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-04-27T07:49:58.411370+0000 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 17 (quincy) 2024-04-27T07:49:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2024-04-27T07:49:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-27T07:49:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-27T07:49:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:2: [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] mon.c 2024-04-27T07:49:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-27T07:49:58.413 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T07:49:58.419 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 807b2bc3-0b23-4239-8346-8619bcda9f4e... 2024-04-27T07:49:58.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T07:49:58.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-04-27T07:49:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-04-27T07:49:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 807b2bc3-0b23-4239-8346-8619bcda9f4e 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-04-27T07:49:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-04-27T07:49:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-04-27T07:49:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-04-27T07:49:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-04-27T07:49:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-04-27T07:49:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: mon op complaint time = 120 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: client mount timeout = 600 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-04-27T07:49:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds = 20 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds balancer = 20 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug frag = True 2024-04-27T07:49:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: mds op complaint time = 180 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2024-04-27T07:49:58.502 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T07:49:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T07:49:58.504 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.22,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = 807b2bc3-0b23-4239-8346-8619bcda9f4e 2024-04-27T07:49:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2024-04-27T07:49:58.506 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug trim objects = true 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2024-04-27T07:49:58.507 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: bdev async discard = True 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: bdev enable discard = True 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore allocator = bitmap 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 1/20 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 1/20 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 4/10 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2024-04-27T07:49:58.508 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2024-04-27T07:49:58.509 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.510 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon down mkfs grace = 300 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: mon op complaint time = 120 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2024-04-27T07:49:58.511 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: client mount timeout = 600 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = "" 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2024-04-27T07:49:58.512 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout:[mds] 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds = 20 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds balancer = 20 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug frag = True 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug scatterstat = True 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: mds op complaint time = 180 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: mds verify scatter = True 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2024-04-27T07:49:58.513 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2024-04-27T07:49:58.515 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-27T07:49:58.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-27T07:49:58.609 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-27T07:49:58.610 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:58.610 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-27T07:49:58.615 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:58.615 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-27T07:49:58.662 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:49:58.663 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:58.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T07:49:58.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:49:58.730 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:49:58.730 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:49:58.779 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:49:58.779 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:58.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T07:49:58.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:49:58.801 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:49:58.801 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:49:58.848 INFO:tasks.ceph:Setting up mon nodes... 2024-04-27T07:49:58.849 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-27T07:49:58.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-27T07:49:58.893 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-27T07:49:58.896 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T07:49:58.939 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T07:49:58.952 INFO:tasks.ceph:Setting up mds nodes... 2024-04-27T07:49:58.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-27T07:49:58.997 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-27T07:49:59.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-27T07:49:59.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-27T07:49:59.141 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-27T07:49:59.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-27T07:49:59.199 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-27T07:49:59.244 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-27T07:49:59.247 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-27T07:49:59.259 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-27T07:49:59.342 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-27T07:49:59.345 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-27T07:49:59.399 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-27T07:49:59.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T07:49:59.437 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-27T07:49:59.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-27T07:49:59.526 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-27T07:49:59.537 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-27T07:49:59.573 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-27T07:49:59.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-27T07:49:59.660 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-27T07:49:59.671 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-27T07:49:59.671 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-27T07:49:59.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-27T07:49:59.686 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:49:59.686 INFO:tasks.ceph:role: osd.0 2024-04-27T07:49:59.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:49:59.686 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:49:59.743 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:49:59.766 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-04-27T07:49:59.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-04-27T07:49:59.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-27T07:49:59.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-27T07:49:59.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:49:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:49:59.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-27T07:49:59.932 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:49:59.932 INFO:tasks.ceph:role: osd.1 2024-04-27T07:49:59.933 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:49:59.933 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T07:49:59.986 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:49:59.987 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:00.010 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-04-27T07:50:00.012 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-04-27T07:50:00.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-27T07:50:00.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-27T07:50:00.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:00.125 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:00.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-27T07:50:00.178 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:00.178 INFO:tasks.ceph:role: osd.2 2024-04-27T07:50:00.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:50:00.179 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:00.235 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:00.259 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-04-27T07:50:00.261 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-04-27T07:50:00.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-27T07:50:00.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-27T07:50:00.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:00.378 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:00.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-27T07:50:00.434 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:00.435 INFO:tasks.ceph:role: osd.3 2024-04-27T07:50:00.435 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:50:00.435 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:00.491 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:00.515 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-04-27T07:50:00.516 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-04-27T07:50:00.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-27T07:50:00.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-27T07:50:00.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:00.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:00.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.745+0000 7f4e731027c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-27T07:50:00.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.745+0000 7f4e731027c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-27T07:50:00.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.745+0000 7f4e731027c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:50:00.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.745+0000 7f4e731027c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:50:00.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.745+0000 7f4e731027c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:50:00.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:00.749+0000 7f4e731027c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-27T07:50:03.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T07:50:03.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:03.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.805+0000 7f671c8497c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-27T07:50:03.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.809+0000 7f671c8497c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-27T07:50:03.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.809+0000 7f671c8497c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:50:03.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.809+0000 7f671c8497c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:50:03.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.809+0000 7f671c8497c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:50:03.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:03.809+0000 7f671c8497c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-27T07:50:06.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T07:50:06.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:06.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.917+0000 7fc9217827c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-27T07:50:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.921+0000 7fc9217827c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-27T07:50:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.921+0000 7fc9217827c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:50:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.921+0000 7fc9217827c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:50:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.921+0000 7fc9217827c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:50:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:06.921+0000 7fc9217827c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-27T07:50:09.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T07:50:09.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:50:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:10.041+0000 7f1be69ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-27T07:50:13.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T07:50:13.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-27T07:50:13.142 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:13.142 INFO:tasks.ceph:role: osd.4 2024-04-27T07:50:13.142 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:50:13.142 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T07:50:13.203 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:13.203 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:13.204 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:13.656 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-04-27T07:50:13.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-04-27T07:50:13.658 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-27T07:50:13.752 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-27T07:50:13.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:13.762 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:13.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-27T07:50:13.814 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:13.814 INFO:tasks.ceph:role: osd.5 2024-04-27T07:50:13.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:50:13.814 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T07:50:13.880 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:13.880 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:13.881 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:14.350 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-04-27T07:50:14.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-04-27T07:50:14.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-27T07:50:14.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-27T07:50:14.520 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:14.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:14.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-27T07:50:14.575 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:14.575 INFO:tasks.ceph:role: osd.6 2024-04-27T07:50:14.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:50:14.575 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T07:50:14.639 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:14.639 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:14.640 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:15.109 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-04-27T07:50:15.111 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-04-27T07:50:15.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-27T07:50:15.242 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-27T07:50:15.253 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:15.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:15.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-27T07:50:15.311 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:50:15.311 INFO:tasks.ceph:role: osd.7 2024-04-27T07:50:15.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:50:15.311 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:50:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:50:15.802 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-04-27T07:50:15.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-04-27T07:50:15.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-27T07:50:15.881 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-27T07:50:15.933 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:50:15.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:50:15.934 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:16.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.030+0000 7fe24df227c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-27T07:50:16.038 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.034+0000 7fe24df227c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-27T07:50:16.038 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.034+0000 7fe24df227c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:50:16.039 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.034+0000 7fe24df227c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:50:16.039 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.034+0000 7fe24df227c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:50:16.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:16.038+0000 7fe24df227c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-27T07:50:19.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T07:50:19.029 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:19.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-27T07:50:19.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-27T07:50:19.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:50:19.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:50:19.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:50:19.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:19.106+0000 7f64667307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-27T07:50:22.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T07:50:22.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:22.227 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.222+0000 7f32117217c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-27T07:50:22.228 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.226+0000 7f32117217c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-27T07:50:22.228 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.226+0000 7f32117217c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:50:22.228 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.226+0000 7f32117217c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:50:22.228 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.226+0000 7f32117217c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:50:22.228 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:22.226+0000 7f32117217c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-27T07:50:25.197 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-27T07:50:25.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-27T07:50:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-27T07:50:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:50:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:50:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:50:25.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:25.330+0000 7f17ad3317c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-27T07:50:26.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:26.750 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:28.316 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-27T07:50:28.329 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-27T07:50:28.330 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-27T07:50:28.344 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-27T07:50:28.398 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-27T07:50:28.458 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-27T07:50:28.518 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-27T07:50:28.574 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-27T07:50:28.630 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:28.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-27T07:50:28.686 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.687 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-27T07:50:28.700 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-27T07:50:28.757 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.757 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-27T07:50:28.810 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.810 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-27T07:50:28.862 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.862 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-27T07:50:28.915 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-27T07:50:28.970 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:28.971 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-27T07:50:29.022 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:29.023 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-27T07:50:29.028 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:29.029 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-27T07:50:29.075 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:29.075 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-27T07:50:29.080 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:29.080 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-27T07:50:29.128 INFO:tasks.ceph:Adding keys to all mons... 2024-04-27T07:50:29.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T07:50:29.130 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T07:50:29.173 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAmrixmXgdLNRAAjhEHY9rn1R6e6o/YiIxO5A== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAmrixmAjl3OxAA96NFo3XfB80lw+5T7bIoGw== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmPrNtCBAA2fbFI8ZI7gmXDQnmz6YHDw== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAorixmHG63LBAAE19Tgew8foVNVEgoA4GTrQ== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQArrixmcnNOMBAAbRIY/UbPaMEI8JZvxbP8Lg== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAurixm4v/5NhAAzHclJ5aq4/QO7MPYYoYxKA== 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2024-04-27T07:50:29.174 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAyrixmMuufAhAAwKgeVoF0TXe1gLsDKVw4LA== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAmrixm8NccOBAA5JurAK+VIdg5PQtbTzIfLg== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixm3NCuDhAA08t9OL34T+b7oRLKgRT8FA== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmWz6DFBAARdqqspUPofRg+F8n27WfPQ== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQA4rixm8Yw6AhAAeUHuDVW0FWW5Iohg+iOxNA== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQA7rixmuWyjBhAAFDNZDS+j+WOczPvGKkuZ1Q== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQA+rixm5imsDRAAo0sn0MkBlRLIvTnCmhGXbQ== 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2024-04-27T07:50:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBBrixmgO8CFBAAOJPLxZIn/68YBTgx+Z6igA== 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmLWgVGhAAj3agIuftac+O2Tov9zpCZQ== 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout:[client.2] 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmawBhHxAArTddaaaUap4fpOevEXznJQ== 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmPaU9IhAAvwk0hgdanfEl/bY5dgaRow== 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout:[client.3] 2024-04-27T07:50:29.176 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmKKl4JxAAgYg/XgMW2yJ6/5PKmiGgDA== 2024-04-27T07:50:29.182 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2024-04-27T07:50:29.182 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAmrixmXgdLNRAAjhEHY9rn1R6e6o/YiIxO5A== 2024-04-27T07:50:29.182 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2024-04-27T07:50:29.182 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAmrixmAjl3OxAA96NFo3XfB80lw+5T7bIoGw== 2024-04-27T07:50:29.182 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmPrNtCBAA2fbFI8ZI7gmXDQnmz6YHDw== 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAorixmHG63LBAAE19Tgew8foVNVEgoA4GTrQ== 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQArrixmcnNOMBAAbRIY/UbPaMEI8JZvxbP8Lg== 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAurixm4v/5NhAAzHclJ5aq4/QO7MPYYoYxKA== 2024-04-27T07:50:29.183 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAyrixmMuufAhAAwKgeVoF0TXe1gLsDKVw4LA== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAmrixm8NccOBAA5JurAK+VIdg5PQtbTzIfLg== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixm3NCuDhAA08t9OL34T+b7oRLKgRT8FA== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmWz6DFBAARdqqspUPofRg+F8n27WfPQ== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA4rixm8Yw6AhAAeUHuDVW0FWW5Iohg+iOxNA== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA7rixmuWyjBhAAFDNZDS+j+WOczPvGKkuZ1Q== 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2024-04-27T07:50:29.184 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA+rixm5imsDRAAo0sn0MkBlRLIvTnCmhGXbQ== 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBBrixmgO8CFBAAOJPLxZIn/68YBTgx+Z6igA== 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmLWgVGhAAj3agIuftac+O2Tov9zpCZQ== 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout:[client.2] 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmawBhHxAArTddaaaUap4fpOevEXznJQ== 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmPaU9IhAAvwk0hgdanfEl/bY5dgaRow== 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout:[client.3] 2024-04-27T07:50:29.185 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQAnrixmKKl4JxAAgYg/XgMW2yJ6/5PKmiGgDA== 2024-04-27T07:50:29.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:50:29.220 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:50:29.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.275 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.322 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.407 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.493 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.580 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:50:29.751 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:50:29.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.835 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.884 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:50:29.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:29.975 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.186 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:50:30.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.271 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.458 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:50:30.536 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-27T07:50:30.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-27T07:50:30.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:50:30.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-27T07:50:30.656 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-27T07:50:30.669 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:50:30.761 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-27T07:50:30.772 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-27T07:50:30.825 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:50:30.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-27T07:50:30.925 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:30.928 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:50:30.971 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-27T07:50:30.971 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-27T07:50:30.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-27T07:50:30.975 INFO:tasks.ceph.mon.a:Started 2024-04-27T07:50:30.975 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-27T07:50:30.975 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-27T07:50:31.017 INFO:tasks.ceph.mon.b:Started 2024-04-27T07:50:31.017 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-27T07:50:31.017 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-27T07:50:31.019 INFO:tasks.ceph.mon.c:Started 2024-04-27T07:50:31.019 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-27T07:50:31.020 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-27T07:50:31.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-27T07:50:31.022 INFO:tasks.ceph.mgr.y:Started 2024-04-27T07:50:31.022 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-27T07:50:31.023 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-27T07:50:31.024 INFO:tasks.ceph.mgr.x:Started 2024-04-27T07:50:31.025 INFO:tasks.ceph:Setting crush tunables to default 2024-04-27T07:50:31.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2024-04-27T07:50:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 Processor -- start 2024-04-27T07:50:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 -- start start 2024-04-27T07:50:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff4070130 con 0x7f3ff40b05b0 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff40d1150 con 0x7f3ff40d0830 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ffa86c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff406bbd0 con 0x7f3ff40b0b00 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff8e09640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3ff40b05b0 0x7f3ff40ce450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35064/0 (socket says 172.21.15.22:35064) 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44614/0 (socket says 172.21.15.22:44614) 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff8e09640 1 -- 172.21.15.22:0/2390954380 learned_addr learned my addr 172.21.15.22:0/2390954380 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 msgr2=0x7f3ff40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 msgr2=0x7f3ff40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 msgr2=0x7f3ff40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 msgr2=0x7f3ff40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:50:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.149+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 msgr2=0x7f3ff40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 msgr2=0x7f3ff40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 msgr2=0x7f3ff40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 msgr2=0x7f3ff40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:50:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:50:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.353+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T07:50:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718744318 0 0) 0x7f3ff4070130 con 0x7f3ff40b05b0 2024-04-27T07:50:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fdc003530 con 0x7f3ff40b05b0 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722363808 0 0) 0x7f3fdc003530 con 0x7f3ff40b05b0 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 msgr2=0x7f3ff40cdf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.183:3300/0 conn(0x7f3ff40b0b00 0x7f3ff40cdf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 msgr2=0x7f3ff40d0c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 --2- 172.21.15.22:0/2390954380 >> v2:172.21.15.22:3300/0 conn(0x7f3ff40d0830 0x7f3ff40d0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40cd9f0 con 0x7f3ff40b05b0 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f3fec001680 con 0x7f3ff40b05b0 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 >> v1:172.21.15.22:6789/0 conn(0x7f3ff40b05b0 legacy=0x7f3ff40ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 0x7f3fdc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc004810 0x7f3fdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc0051d0 0x7f3fdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40d1150 con 0x7f3fdc003a10 2024-04-27T07:50:31.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff406bbd0 con 0x7f3fdc004810 2024-04-27T07:50:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.377+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc00c120 con 0x7f3fdc0051d0 2024-04-27T07:50:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 0x7f3fdc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc004810 0x7f3fdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff8e09640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc0051d0 0x7f3fdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc0051d0 msgr2=0x7f3fdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc0051d0 0x7f3fdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc004810 msgr2=0x7f3fdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc004810 0x7f3fdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff3fff640 1 -- 172.21.15.22:0/2390954380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc003530 con 0x7f3fdc003a10 2024-04-27T07:50:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.381+0000 7f3ff8e09640 1 -- 172.21.15.22:0/2390954380 reap_dead start 2024-04-27T07:50:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ff3fff640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 0x7f3fdc003e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009a40 tx=0x7f3fe401cff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253cee6b719e9712 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:31.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b2070 con 0x7f3fdc003a10 2024-04-27T07:50:31.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe40a5dc0 con 0x7f3fdc003a10 2024-04-27T07:50:31.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ff2ffd640 1 -- 172.21.15.22:0/2390954380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4009640 con 0x7f3fdc003a10 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 msgr2=0x7f3fdc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.385+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 0x7f3fdc003e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009a40 tx=0x7f3fe401cff0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.389+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2390954380 shutdown_connections 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.389+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2390954380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fdc003a10 0x7f3fdc003e30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.389+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2390954380 >> 172.21.15.22:0/2390954380 conn(0x7f3ff409aa30 msgr2=0x7f3ff40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.389+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2390954380 shutdown_connections 2024-04-27T07:50:31.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.389+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2390954380 wait complete. 2024-04-27T07:50:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 Processor -- start 2024-04-27T07:50:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 -- start start 2024-04-27T07:50:31.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 0x7f3ff40b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff40c7040 0x7f3ff40c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40cd650 con 0x7f3ff40b3930 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40cbb50 con 0x7f3ff40c7040 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ffa86c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40cd7d0 con 0x7f3fdc003a10 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 0x7f3ff40b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff8e09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff40c7040 0x7f3ff40c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44634/0 (socket says 172.21.15.22:44634) 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2356804651 learned_addr learned my addr 172.21.15.22:0/2356804651 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 0x7f3ff40b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:57858/0 (socket says 172.21.15.22:57858) 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 msgr2=0x7f3ff40b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 0x7f3ff40b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff40c7040 msgr2=0x7f3ff40c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff40c7040 0x7f3ff40c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.393+0000 7f3ff37fe640 1 -- 172.21.15.22:0/2356804651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff41fc3f0 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff37fe640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8013750 tx=0x7f3fe8013c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe801a070 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe8011c50 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41fc6d0 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800e050 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41fcf90 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3fe801d070 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3fe801f900 con 0x7f3ff40b3930 2024-04-27T07:50:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.397+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff40c7a00 con 0x7f3ff40b3930 2024-04-27T07:50:31.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.401+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3fe8016460 con 0x7f3ff40b3930 2024-04-27T07:50:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.465+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3ff404ff10 con 0x7f3ff40b3930 2024-04-27T07:50:31.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ff0ff9640 1 -- 172.21.15.22:0/2356804651 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3fe801fb00 con 0x7f3ff40b3930 2024-04-27T07:50:31.471 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-04-27T07:50:31.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 msgr2=0x7f3ff41fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8013750 tx=0x7f3fe8013c10 comp rx=0 tx=0).stop 2024-04-27T07:50:31.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 shutdown_connections 2024-04-27T07:50:31.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff40c7040 0x7f3ff40c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff40b3930 0x7f3ff41fbbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 --2- 172.21.15.22:0/2356804651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc003a10 0x7f3ff40b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 >> 172.21.15.22:0/2356804651 conn(0x7f3ff409aa30 msgr2=0x7f3ff415c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 shutdown_connections 2024-04-27T07:50:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:31.469+0000 7f3ffa86c640 1 -- 172.21.15.22:0/2356804651 wait complete. 2024-04-27T07:50:31.491 INFO:tasks.ceph:check_enable_crimson: False 2024-04-27T07:50:31.491 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-27T07:50:31.491 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:31.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-27T07:50:31.503 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:31.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-27T07:50:31.561 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:31.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-27T07:50:31.617 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:50:31.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-27T07:50:31.619 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:31.617+0000 7fa3f1740180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T07:50:31.630 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:31.630 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-27T07:50:31.641 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:31.638+0000 7f401ddd4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T07:50:31.644 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:31.644 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-27T07:50:31.698 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:31.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-27T07:50:31.751 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:50:31.752 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-27T07:50:31.807 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 3907e5f3-d06c-49c1-a198-583478c871ce 0 2024-04-27T07:50:31.974 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.970+0000 7f13d0f2d640 1 Processor -- start 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.970+0000 7f13d0f2d640 1 -- start start 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13d0f2d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f13cc0d0670 0x7f13cc167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13d0f2d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc06fba0 con 0x7f13cc0cfb80 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13d0f2d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc06fe40 con 0x7f13cc06ecd0 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13d0f2d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc070120 con 0x7f13cc0d0670 2024-04-27T07:50:31.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c9d9b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f13cc06ecd0 0x7f13cc0cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:53094/0 (socket says 172.21.15.183:53094) 2024-04-27T07:50:31.976 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c9d9b640 1 -- 172.21.15.183:0/2386412630 learned_addr learned my addr 172.21.15.183:0/2386412630 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-04-27T07:50:31.977 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882425684 0 0) 0x7f13cc06fe40 con 0x7f13cc06ecd0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 --2- 172.21.15.183:0/2386412630 >> v2:172.21.15.22:3300/0 conn(0x7f13cc0d0670 0x7f13cc167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b4003530 con 0x7f13cc06ecd0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3562064418 0 0) 0x7f13b4003530 con 0x7f13cc06ecd0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 >> v2:172.21.15.22:3300/0 conn(0x7f13cc0d0670 msgr2=0x7f13cc167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 --2- 172.21.15.183:0/2386412630 >> v2:172.21.15.22:3300/0 conn(0x7f13cc0d0670 0x7f13cc167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 >> v1:172.21.15.22:6789/0 conn(0x7f13cc0cfb80 legacy=0x7f13cc15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc06bdb0 con 0x7f13cc06ecd0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f13c0001680 con 0x7f13cc06ecd0 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 >> v1:172.21.15.183:6789/0 conn(0x7f13cc06ecd0 legacy=0x7f13cc0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13b40039c0 0x7f13b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13b4004810 0x7f13b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.978 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc070120 con 0x7f13b4004810 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400c1c0 con 0x7f13b40052f0 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f13b400c3c0 con 0x7f13b40039c0 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13ca59c640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13b40039c0 0x7f13b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13c9d9b640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13b4004810 0x7f13b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 -- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13b40039c0 msgr2=0x7f13b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13b40039c0 0x7f13b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 -- 172.21.15.183:0/2386412630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13b4004810 msgr2=0x7f13b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13b4004810 0x7f13b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.979 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 -- 172.21.15.183:0/2386412630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4003530 con 0x7f13b40052f0 2024-04-27T07:50:31.980 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 -- 172.21.15.183:0/2386412630 reap_dead start 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.974+0000 7f13cad9d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13b400b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009fc0 tx=0x7f13bc01ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afbcab0c58197e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc009bc0 con 0x7f13b40052f0 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc0a5c30 con 0x7f13b40052f0 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13c959a640 1 -- 172.21.15.183:0/2386412630 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0ad440 con 0x7f13b40052f0 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 msgr2=0x7f13b400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13b400b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009fc0 tx=0x7f13bc01ce40 comp rx=0 tx=0).stop 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/2386412630 shutdown_connections 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/2386412630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13b400b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/2386412630 >> 172.21.15.183:0/2386412630 conn(0x7f13cc0b32c0 msgr2=0x7f13cc0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/2386412630 shutdown_connections 2024-04-27T07:50:31.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/2386412630 wait complete. 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 Processor -- start 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- start start 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13cc150cc0 0x7f13cc151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13cc151a90 0x7f13cc1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0d02f0 con 0x7f13cc151a90 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0d0170 con 0x7f13b40052f0 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc0d0470 con 0x7f13cc150cc0 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59584/0 (socket says 172.21.15.183:59584) 2024-04-27T07:50:31.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 -- 172.21.15.183:0/3490505751 learned_addr learned my addr 172.21.15.183:0/3490505751 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13c9d9b640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13cc150cc0 0x7f13cc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13cad9d640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13cc151a90 0x7f13cc1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 -- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13cc150cc0 msgr2=0x7f13cc151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13cc150cc0 0x7f13cc151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 -- 172.21.15.183:0/3490505751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13cc151a90 msgr2=0x7f13cc1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13cc151a90 0x7f13cc1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 -- 172.21.15.183:0/3490505751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc14fc20 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13ca59c640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13c400f5f0 tx=0x7f13c400fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40052e0 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c4018070 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4003820 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc14ff00 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.978+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1504c0 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.982+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f13c40039c0 con 0x7f13b40052f0 2024-04-27T07:50:31.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.982+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f13c401dd00 con 0x7f13b40052f0 2024-04-27T07:50:31.986 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.982+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc04ff10 con 0x7f13b40052f0 2024-04-27T07:50:31.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:31.986+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f13c4005480 con 0x7f13b40052f0 2024-04-27T07:50:32.063 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.058+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3907e5f3-d06c-49c1-a198-583478c871ce", "id": 0} v 0) -- 0x7f13cc0d1920 con 0x7f13b40052f0 2024-04-27T07:50:32.067 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.062+0000 7f13baffd640 1 -- 172.21.15.183:0/3490505751 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3907e5f3-d06c-49c1-a198-583478c871ce", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f13c4003df0 con 0x7f13b40052f0 2024-04-27T07:50:32.067 INFO:teuthology.orchestra.run.smithi183.stdout:0 2024-04-27T07:50:32.068 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 msgr2=0x7f13cc14def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13c400f5f0 tx=0x7f13c400fab0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 shutdown_connections 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13cc151a90 0x7f13cc1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13cc150cc0 0x7f13cc151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 --2- 172.21.15.183:0/3490505751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b40052f0 0x7f13cc14def0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 >> 172.21.15.183:0/3490505751 conn(0x7f13cc0b32c0 msgr2=0x7f13cc0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 shutdown_connections 2024-04-27T07:50:32.069 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.066+0000 7f13d0f2d640 1 -- 172.21.15.183:0/3490505751 wait complete. 2024-04-27T07:50:32.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 902b6447-8080-487a-b5fb-12222688f566 1 2024-04-27T07:50:32.148 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:32.145+0000 7fa3f1740180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T07:50:32.149 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:32.146+0000 7f401ddd4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T07:50:32.206 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 Processor -- start 2024-04-27T07:50:32.206 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 -- start start 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff81c0d0690 0x7ff81c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c0b4440 con 0x7ff81c0cfae0 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c0b4720 con 0x7ff81c0cef10 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff821a20640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff81c0b4a00 con 0x7ff81c0d0690 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff81a7fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff81c0d0690 0x7ff81c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff81a7fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff81c0d0690 0x7ff81c0ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59598/0 (socket says 172.21.15.183:59598) 2024-04-27T07:50:32.208 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.202+0000 7ff81a7fc640 1 -- 172.21.15.183:0/2724582174 learned_addr learned my addr 172.21.15.183:0/2724582174 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.208 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff81a7fc640 1 -- 172.21.15.183:0/2724582174 >> v1:172.21.15.183:6789/0 conn(0x7ff81c0cef10 legacy=0x7ff81c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.208 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff81a7fc640 1 -- 172.21.15.183:0/2724582174 >> v1:172.21.15.22:6789/0 conn(0x7ff81c0cfae0 legacy=0x7ff81c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.208 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff81a7fc640 1 -- 172.21.15.183:0/2724582174 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c0b4f40 con 0x7ff81c0d0690 2024-04-27T07:50:32.209 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff81a7fc640 1 --2- 172.21.15.183:0/2724582174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0d0690 0x7ff81c0ce9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff804009870 tx=0x7ff80401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a5fa37b1065aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.209 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff819ffb640 1 -- 172.21.15.183:0/2724582174 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040b0070 con 0x7ff81c0d0690 2024-04-27T07:50:32.209 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff819ffb640 1 -- 172.21.15.183:0/2724582174 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8040ab600 con 0x7ff81c0d0690 2024-04-27T07:50:32.210 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff819ffb640 1 -- 172.21.15.183:0/2724582174 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040ab9d0 con 0x7ff81c0d0690 2024-04-27T07:50:32.210 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 -- 172.21.15.183:0/2724582174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0d0690 msgr2=0x7ff81c0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 --2- 172.21.15.183:0/2724582174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0d0690 0x7ff81c0ce9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff804009870 tx=0x7ff80401c4d0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 -- 172.21.15.183:0/2724582174 shutdown_connections 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 --2- 172.21.15.183:0/2724582174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0d0690 0x7ff81c0ce9d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 -- 172.21.15.183:0/2724582174 >> 172.21.15.183:0/2724582174 conn(0x7ff81c0b32c0 msgr2=0x7ff81c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 -- 172.21.15.183:0/2724582174 shutdown_connections 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.206+0000 7ff821a20640 1 -- 172.21.15.183:0/2724582174 wait complete. 2024-04-27T07:50:32.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 Processor -- start 2024-04-27T07:50:32.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- start start 2024-04-27T07:50:32.213 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.213 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0cfae0 0x7ff81c160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.213 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff81c0d0690 0x7ff81c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c15f980 con 0x7ff81c0d0690 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c15f800 con 0x7ff81c0cfae0 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c1694a0 con 0x7ff81c0cef10 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:33874/0 (socket says 172.21.15.183:33874) 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 -- 172.21.15.183:0/2502458650 learned_addr learned my addr 172.21.15.183:0/2502458650 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81a7fc640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0cfae0 0x7ff81c160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81b7fe640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff81c0d0690 0x7ff81c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 -- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0cfae0 msgr2=0x7ff81c160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0cfae0 0x7ff81c160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 -- 172.21.15.183:0/2502458650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff81c0d0690 msgr2=0x7ff81c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.214 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff81c0d0690 0x7ff81c1551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.215 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 -- 172.21.15.183:0/2502458650 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1570a0 con 0x7ff81c0cef10 2024-04-27T07:50:32.215 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff81affd640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c500 tx=0x7ff81400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.215 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814016020 con 0x7ff81c0cef10 2024-04-27T07:50:32.215 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814005220 con 0x7ff81c0cef10 2024-04-27T07:50:32.215 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814007030 con 0x7ff81c0cef10 2024-04-27T07:50:32.216 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c155950 con 0x7ff81c0cef10 2024-04-27T07:50:32.216 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c157360 con 0x7ff81c0cef10 2024-04-27T07:50:32.216 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8140067e0 con 0x7ff81c0cef10 2024-04-27T07:50:32.216 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7ff814012cf0 con 0x7ff81c0cef10 2024-04-27T07:50:32.216 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.210+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c04ff10 con 0x7ff81c0cef10 2024-04-27T07:50:32.217 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.214+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff814020020 con 0x7ff81c0cef10 2024-04-27T07:50:32.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.282+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "902b6447-8080-487a-b5fb-12222688f566", "id": 1} v 0) -- 0x7ff81c0d1890 con 0x7ff81c0cef10 2024-04-27T07:50:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff813fff640 1 -- 172.21.15.183:0/2502458650 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "902b6447-8080-487a-b5fb-12222688f566", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7ff814007580 con 0x7ff81c0cef10 2024-04-27T07:50:32.292 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 msgr2=0x7ff81c157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c500 tx=0x7ff81400c9c0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 shutdown_connections 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff81c0d0690 0x7ff81c1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0cfae0 0x7ff81c160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 --2- 172.21.15.183:0/2502458650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c0cef10 0x7ff81c157b60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.293 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 >> 172.21.15.183:0/2502458650 conn(0x7ff81c0b32c0 msgr2=0x7ff81c15cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.294 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 shutdown_connections 2024-04-27T07:50:32.294 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.290+0000 7ff821a20640 1 -- 172.21.15.183:0/2502458650 wait complete. 2024-04-27T07:50:32.309 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new c853f889-92ce-4a6f-8f89-475e7ccb18b4 2 2024-04-27T07:50:32.343 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:32.341+0000 7fa3f1740180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T07:50:32.358 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:32.354+0000 7f401ddd4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T07:50:32.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 Processor -- start 2024-04-27T07:50:32.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 -- start start 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fca9c15b750 0x7fca9c0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9c0b2730 con 0x7fca9c0cd920 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9c0b2a10 con 0x7fca9c0d1b40 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa36ea640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fca9c0cf800 con 0x7fca9c15b750 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa1486640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fca9c15b750 0x7fca9c0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa1486640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fca9c15b750 0x7fca9c0d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59616/0 (socket says 172.21.15.183:59616) 2024-04-27T07:50:32.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.422+0000 7fcaa1486640 1 -- 172.21.15.183:0/403623756 learned_addr learned my addr 172.21.15.183:0/403623756 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.428 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa1486640 1 -- 172.21.15.183:0/403623756 >> v1:172.21.15.183:6789/0 conn(0x7fca9c0d1b40 legacy=0x7fca9c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.428 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa1486640 1 -- 172.21.15.183:0/403623756 >> v1:172.21.15.22:6789/0 conn(0x7fca9c0cd920 legacy=0x7fca9c0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.428 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa1486640 1 -- 172.21.15.183:0/403623756 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c0b2cf0 con 0x7fca9c15b750 2024-04-27T07:50:32.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa1486640 1 --2- 172.21.15.183:0/403623756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c0d0c00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca98009870 tx=0x7fca9801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fe40e7d4b5d65a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fca93fff640 1 -- 172.21.15.183:0/403623756 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980b0070 con 0x7fca9c15b750 2024-04-27T07:50:32.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fca93fff640 1 -- 172.21.15.183:0/403623756 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca980ab600 con 0x7fca9c15b750 2024-04-27T07:50:32.430 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fca93fff640 1 -- 172.21.15.183:0/403623756 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980ab9d0 con 0x7fca9c15b750 2024-04-27T07:50:32.430 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 -- 172.21.15.183:0/403623756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 msgr2=0x7fca9c0d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.430 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/403623756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c0d0c00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca98009870 tx=0x7fca9801c520 comp rx=0 tx=0).stop 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 -- 172.21.15.183:0/403623756 shutdown_connections 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/403623756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c0d0c00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 -- 172.21.15.183:0/403623756 >> 172.21.15.183:0/403623756 conn(0x7fca9c0b68a0 msgr2=0x7fca9c0b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 -- 172.21.15.183:0/403623756 shutdown_connections 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.426+0000 7fcaa36ea640 1 -- 172.21.15.183:0/403623756 wait complete. 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 Processor -- start 2024-04-27T07:50:32.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- start start 2024-04-27T07:50:32.433 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c0cd920 0x7fca9c14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.433 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 0x7fca9c15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c169710 con 0x7fca9c0cd920 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c169590 con 0x7fca9c15b750 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c169890 con 0x7fca9c0d1b40 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa0c85640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 0x7fca9c15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59632/0 (socket says 172.21.15.183:59632) 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 -- 172.21.15.183:0/1426611069 learned_addr learned my addr 172.21.15.183:0/1426611069 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1486640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c0cd920 0x7fca9c14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 -- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 msgr2=0x7fca9c15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 0x7fca9c15a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 -- 172.21.15.183:0/1426611069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c0cd920 msgr2=0x7fca9c14d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c0cd920 0x7fca9c14d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 -- 172.21.15.183:0/1426611069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c15a650 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa0c85640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 0x7fca9c15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa1c87640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca9400c770 tx=0x7fca9400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94003b80 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca94003d20 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c15a8f0 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400aa90 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c0b4660 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca9400ac30 con 0x7fca9c15b750 2024-04-27T07:50:32.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c04ff10 con 0x7fca9c15b750 2024-04-27T07:50:32.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.430+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fca940054f0 con 0x7fca9c15b750 2024-04-27T07:50:32.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.434+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fca940050d0 con 0x7fca9c15b750 2024-04-27T07:50:32.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.498+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c853f889-92ce-4a6f-8f89-475e7ccb18b4", "id": 2} v 0) -- 0x7fca9c154b90 con 0x7fca9c15b750 2024-04-27T07:50:32.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fca91ffb640 1 -- 172.21.15.183:0/1426611069 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c853f889-92ce-4a6f-8f89-475e7ccb18b4", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fca940125b0 con 0x7fca9c15b750 2024-04-27T07:50:32.508 INFO:teuthology.orchestra.run.smithi183.stdout:2 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 msgr2=0x7fca9c152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca9400c770 tx=0x7fca9400cc30 comp rx=0 tx=0).stop 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 shutdown_connections 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca9c15b750 0x7fca9c152920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca9c0d1b40 0x7fca9c15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 --2- 172.21.15.183:0/1426611069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c0cd920 0x7fca9c14d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 >> 172.21.15.183:0/1426611069 conn(0x7fca9c0b68a0 msgr2=0x7fca9c070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 shutdown_connections 2024-04-27T07:50:32.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.506+0000 7fcaa36ea640 1 -- 172.21.15.183:0/1426611069 wait complete. 2024-04-27T07:50:32.525 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 91b7cfd1-6847-407c-b6be-6be0e93bbd83 3 2024-04-27T07:50:32.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.630+0000 7f3029524640 1 Processor -- start 2024-04-27T07:50:32.635 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.630+0000 7f3029524640 1 -- start start 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240cd150 con 0x7f302415fdc0 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240d1560 con 0x7f30240b3440 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f30240d0cc0 con 0x7f302409b140 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30227fc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f30240b3440 0x7f302409aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:53128/0 (socket says 172.21.15.183:53128) 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30227fc640 1 -- 172.21.15.183:0/2309588463 learned_addr learned my addr 172.21.15.183:0/2309588463 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:33888/0 (socket says 172.21.15.183:33888) 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 -- 172.21.15.183:0/2309588463 >> v1:172.21.15.183:6789/0 conn(0x7f30240b3440 legacy=0x7f302409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.636 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 -- 172.21.15.183:0/2309588463 >> v1:172.21.15.22:6789/0 conn(0x7f302415fdc0 legacy=0x7f302409b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.637 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 -- 172.21.15.183:0/2309588463 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240d0fa0 con 0x7f302409b140 2024-04-27T07:50:32.637 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3021ffb640 1 -- 172.21.15.183:0/2309588463 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080297848 0 0) 0x7f30240d1560 con 0x7f30240b3440 2024-04-27T07:50:32.638 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f30237fe640 1 --2- 172.21.15.183:0/2309588463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f301c009dc0 tx=0x7f301c01c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6457d6d674558cad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.638 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3021ffb640 1 -- 172.21.15.183:0/2309588463 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0b2070 con 0x7f302409b140 2024-04-27T07:50:32.638 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3021ffb640 1 -- 172.21.15.183:0/2309588463 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c0a5800 con 0x7f302409b140 2024-04-27T07:50:32.638 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3021ffb640 1 -- 172.21.15.183:0/2309588463 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0a5bd0 con 0x7f302409b140 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- 172.21.15.183:0/2309588463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 msgr2=0x7f30240b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 --2- 172.21.15.183:0/2309588463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f301c009dc0 tx=0x7f301c01c8c0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- 172.21.15.183:0/2309588463 shutdown_connections 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 --2- 172.21.15.183:0/2309588463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240b5490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- 172.21.15.183:0/2309588463 >> 172.21.15.183:0/2309588463 conn(0x7f30240b4130 msgr2=0x7f30240b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- 172.21.15.183:0/2309588463 shutdown_connections 2024-04-27T07:50:32.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.634+0000 7f3029524640 1 -- 172.21.15.183:0/2309588463 wait complete. 2024-04-27T07:50:32.640 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 Processor -- start 2024-04-27T07:50:32.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- start start 2024-04-27T07:50:32.643 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.643 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 0x7f302414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302415fdc0 0x7f30241566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30240ce530 con 0x7f302415fdc0 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30240ce3b0 con 0x7f30240b3440 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f302416a800 con 0x7f302409b140 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f30227fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 0x7f302414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:33900/0 (socket says 172.21.15.183:33900) 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 -- 172.21.15.183:0/406764864 learned_addr learned my addr 172.21.15.183:0/406764864 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f30237fe640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302415fdc0 0x7f30241566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 -- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 msgr2=0x7f302414cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 0x7f302414cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.644 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 -- 172.21.15.183:0/406764864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302415fdc0 msgr2=0x7f30241566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302415fdc0 0x7f30241566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 -- 172.21.15.183:0/406764864 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3024156f40 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f30227fc640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 0x7f302414cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3022ffd640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f301800b440 tx=0x7f301800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018015020 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3018011890 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3024157220 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018011c80 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.638+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241577e0 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.642+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3018018020 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.642+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f301801a890 con 0x7f302409b140 2024-04-27T07:50:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.642+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302404ff10 con 0x7f302409b140 2024-04-27T07:50:32.646 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.642+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f301801f030 con 0x7f302409b140 2024-04-27T07:50:32.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.710+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "91b7cfd1-6847-407c-b6be-6be0e93bbd83", "id": 3} v 0) -- 0x7f30240716f0 con 0x7f302409b140 2024-04-27T07:50:32.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f300b7fe640 1 -- 172.21.15.183:0/406764864 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91b7cfd1-6847-407c-b6be-6be0e93bbd83", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f301800d040 con 0x7f302409b140 2024-04-27T07:50:32.720 INFO:teuthology.orchestra.run.smithi183.stdout:3 2024-04-27T07:50:32.721 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 msgr2=0x7f30240cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f301800b440 tx=0x7f301800b900 comp rx=0 tx=0).stop 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 shutdown_connections 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302415fdc0 0x7f30241566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240b3440 0x7f302414cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 --2- 172.21.15.183:0/406764864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302409b140 0x7f30240cbe00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 >> 172.21.15.183:0/406764864 conn(0x7f30240b4130 msgr2=0x7f302409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 shutdown_connections 2024-04-27T07:50:32.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.718+0000 7f3029524640 1 -- 172.21.15.183:0/406764864 wait complete. 2024-04-27T07:50:32.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new dee730ac-0187-42fd-a626-0bf08ea54b70 4 2024-04-27T07:50:32.854 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 Processor -- start 2024-04-27T07:50:32.854 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 -- start start 2024-04-27T07:50:32.857 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f378806c2e0 0x7f3788167470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.857 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378815dc10 con 0x7f3788160dd0 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37880d1300 con 0x7f378815a370 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.850+0000 7f379029a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f378815a750 con 0x7f378806c2e0 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e036640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f378815a370 0x7f37880d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:39714/0 (socket says 172.21.15.183:39714) 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e036640 1 -- 172.21.15.183:0/58016355 learned_addr learned my addr 172.21.15.183:0/58016355 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e837640 1 --2- 172.21.15.183:0/58016355 >> v2:172.21.15.22:3300/0 conn(0x7f378806c2e0 0x7f3788167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197923392 0 0) 0x7f37880d1300 con 0x7f378815a370 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3774003530 con 0x7f378815a370 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 309419698 0 0) 0x7f3774003530 con 0x7f378815a370 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 >> v2:172.21.15.22:3300/0 conn(0x7f378806c2e0 msgr2=0x7f3788167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 --2- 172.21.15.183:0/58016355 >> v2:172.21.15.22:3300/0 conn(0x7f378806c2e0 0x7f3788167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 >> v1:172.21.15.22:6789/0 conn(0x7f3788160dd0 legacy=0x7f378806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.858 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37880d23d0 con 0x7f378815a370 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f3780001a00 con 0x7f378815a370 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 >> v1:172.21.15.183:6789/0 conn(0x7f378815a370 legacy=0x7f37880d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37740039c0 0x7f3774003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 0x7f3774004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37740051d0 0x7f377400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f378815a750 con 0x7f37740039c0 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f377400c170 con 0x7f3774004810 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f377400c370 con 0x7f37740051d0 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 0x7f3774004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e837640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37740051d0 0x7f377400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 -- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37740051d0 msgr2=0x7f377400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37740051d0 0x7f377400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 -- 172.21.15.183:0/58016355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37740039c0 msgr2=0x7f3774003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:32.859 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37740039c0 0x7f3774003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.860 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378d835640 1 -- 172.21.15.183:0/58016355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3774003530 con 0x7f3774004810 2024-04-27T07:50:32.860 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e837640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37740051d0 0x7f377400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:32.860 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.854+0000 7f378e837640 1 -- 172.21.15.183:0/58016355 reap_dead start 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378d835640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 0x7f3774004c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3778001780 tx=0x7f377801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c5b057f86209690 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780013a0 con 0x7f3774004810 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37780052f0 con 0x7f3774004810 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378d034640 1 -- 172.21.15.183:0/58016355 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778003840 con 0x7f3774004810 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 msgr2=0x7f3774004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 0x7f3774004c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3778001780 tx=0x7f377801c400 comp rx=0 tx=0).stop 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- 172.21.15.183:0/58016355 shutdown_connections 2024-04-27T07:50:32.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 --2- 172.21.15.183:0/58016355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3774004810 0x7f3774004c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- 172.21.15.183:0/58016355 >> 172.21.15.183:0/58016355 conn(0x7f378809aa30 msgr2=0x7f37880d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- 172.21.15.183:0/58016355 shutdown_connections 2024-04-27T07:50:32.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- 172.21.15.183:0/58016355 wait complete. 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 Processor -- start 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- start start 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37881f8010 0x7f37880c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f378806f0b0 0x7f378806f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:32.864 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f378806bfe0 con 0x7f378806f0b0 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3788161200 con 0x7f37881f8010 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f379029a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f378806c160 con 0x7f3774004810 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:35550/0 (socket says 172.21.15.183:35550) 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 -- 172.21.15.183:0/2970331855 learned_addr learned my addr 172.21.15.183:0/2970331855 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378d835640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37881f8010 0x7f37880c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e837640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f378806f0b0 0x7f378806f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 -- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37881f8010 msgr2=0x7f37880c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37881f8010 0x7f37880c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 -- 172.21.15.183:0/2970331855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f378806f0b0 msgr2=0x7f378806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f378806f0b0 0x7f378806f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.858+0000 7f378e036640 1 -- 172.21.15.183:0/2970331855 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881f87a0 con 0x7f3774004810 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f378e036640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f378000cb20 tx=0x7f3780001ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:32.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780018070 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378000a760 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881f8a80 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378000ab50 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881f8490 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f378000acf0 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3780012be0 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378804ff10 con 0x7f3774004810 2024-04-27T07:50:32.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.862+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3780021070 con 0x7f3774004810 2024-04-27T07:50:32.935 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.930+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dee730ac-0187-42fd-a626-0bf08ea54b70", "id": 4} v 0) -- 0x7f37880af450 con 0x7f3774004810 2024-04-27T07:50:32.938 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:32.934+0000 7f401ddd4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T07:50:32.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f377effd640 1 -- 172.21.15.183:0/2970331855 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dee730ac-0187-42fd-a626-0bf08ea54b70", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f37800126d0 con 0x7f3774004810 2024-04-27T07:50:32.941 INFO:teuthology.orchestra.run.smithi183.stdout:4 2024-04-27T07:50:32.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 msgr2=0x7f3788070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:32.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f378000cb20 tx=0x7f3780001ae0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 shutdown_connections 2024-04-27T07:50:32.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f378806f0b0 0x7f378806f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37881f8010 0x7f37880c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 --2- 172.21.15.183:0/2970331855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3774004810 0x7f3788070c00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:32.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 >> 172.21.15.183:0/2970331855 conn(0x7f378809aa30 msgr2=0x7f378815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:32.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 shutdown_connections 2024-04-27T07:50:32.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:32.938+0000 7f379029a640 1 -- 172.21.15.183:0/2970331855 wait complete. 2024-04-27T07:50:32.944 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:32.941+0000 7fa3f1740180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T07:50:32.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 89452ade-5696-4d0b-98e0-af57d4f928e9 5 2024-04-27T07:50:32.995 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:32.994+0000 7f401ddd4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T07:50:32.999 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:32.997+0000 7fa3f1740180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T07:50:33.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.046+0000 7f401ddd4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T07:50:33.054 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.053+0000 7fa3f1740180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T07:50:33.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.070+0000 7fb8bee1c640 1 Processor -- start 2024-04-27T07:50:33.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.070+0000 7fb8bee1c640 1 -- start start 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 0x7fb8b80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b80b4440 con 0x7fb8b815f0f0 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b80b4720 con 0x7fb8b80d0690 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b80b4a00 con 0x7fb8b80cfae0 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bcbb8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 0x7fb8b80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b7fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fb8b80d0690 0x7fb8b80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:39718/0 (socket says 172.21.15.183:39718) 2024-04-27T07:50:33.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b7fff640 1 -- 172.21.15.183:0/3921401096 learned_addr learned my addr 172.21.15.183:0/3921401096 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-04-27T07:50:33.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/3921401096 >> v1:172.21.15.183:6789/0 conn(0x7fb8b80d0690 legacy=0x7fb8b80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/3921401096 >> v1:172.21.15.22:6789/0 conn(0x7fb8b815f0f0 legacy=0x7fb8b80b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b77fe640 1 -- 172.21.15.183:0/3921401096 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660741839 0 0) 0x7fb8b80b4720 con 0x7fb8b80d0690 2024-04-27T07:50:33.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/3921401096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b80b4f40 con 0x7fb8b80cfae0 2024-04-27T07:50:33.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bcbb8640 1 --2- 172.21.15.183:0/3921401096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 0x7fb8b80b3b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009dc0 tx=0x7fb8b001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1845e670c920ca37 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b77fe640 1 -- 172.21.15.183:0/3921401096 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00b2070 con 0x7fb8b80cfae0 2024-04-27T07:50:33.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b77fe640 1 -- 172.21.15.183:0/3921401096 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b00a58d0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8b77fe640 1 -- 172.21.15.183:0/3921401096 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00a5ca0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- 172.21.15.183:0/3921401096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 msgr2=0x7fb8b80b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/3921401096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 0x7fb8b80b3b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009dc0 tx=0x7fb8b001c900 comp rx=0 tx=0).stop 2024-04-27T07:50:33.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- 172.21.15.183:0/3921401096 shutdown_connections 2024-04-27T07:50:33.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/3921401096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80cfae0 0x7fb8b80b3b80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- 172.21.15.183:0/3921401096 >> 172.21.15.183:0/3921401096 conn(0x7fb8b80b32c0 msgr2=0x7fb8b80d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- 172.21.15.183:0/3921401096 shutdown_connections 2024-04-27T07:50:33.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.074+0000 7fb8bee1c640 1 -- 172.21.15.183:0/3921401096 wait complete. 2024-04-27T07:50:33.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 Processor -- start 2024-04-27T07:50:33.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- start start 2024-04-27T07:50:33.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 0x7fb8b8158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8b815f0f0 0x7fb8b8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8169e30 con 0x7fb8b815f0f0 2024-04-27T07:50:33.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8169cb0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b8169fb0 con 0x7fb8b80d0690 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38570/0 (socket says 172.21.15.183:38570) 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/617139301 learned_addr learned my addr 172.21.15.183:0/617139301 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b7fff640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 0x7fb8b8158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bd3b9640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8b815f0f0 0x7fb8b8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 msgr2=0x7fb8b8158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 0x7fb8b8158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/617139301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8b815f0f0 msgr2=0x7fb8b8154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8b815f0f0 0x7fb8b8154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 -- 172.21.15.183:0/617139301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b814fdd0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b7fff640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 0x7fb8b8158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bcbb8640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00b0040 tx=0x7fb8b00196a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00b2040 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b00bf070 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00b88b0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b81500b0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b8150670 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8b00039a0 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb8b00ad440 con 0x7fb8b80cfae0 2024-04-27T07:50:33.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.078+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b804ff10 con 0x7fb8b80cfae0 2024-04-27T07:50:33.086 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.082+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb8b00c8020 con 0x7fb8b80cfae0 2024-04-27T07:50:33.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.130+0000 7f401ddd4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T07:50:33.134 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.133+0000 7fa3f1740180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T07:50:33.154 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.150+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89452ade-5696-4d0b-98e0-af57d4f928e9", "id": 5} v 0) -- 0x7fb8b80d1890 con 0x7fb8b80cfae0 2024-04-27T07:50:33.159 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.154+0000 7fb8b57fa640 1 -- 172.21.15.183:0/617139301 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89452ade-5696-4d0b-98e0-af57d4f928e9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb8b0016580 con 0x7fb8b80cfae0 2024-04-27T07:50:33.159 INFO:teuthology.orchestra.run.smithi183.stdout:5 2024-04-27T07:50:33.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 msgr2=0x7fb8b814f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00b0040 tx=0x7fb8b00196a0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 shutdown_connections 2024-04-27T07:50:33.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8b815f0f0 0x7fb8b8154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8b80d0690 0x7fb8b8158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 --2- 172.21.15.183:0/617139301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b80cfae0 0x7fb8b814f2e0 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00b0040 tx=0x7fb8b00196a0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 >> 172.21.15.183:0/617139301 conn(0x7fb8b80b32c0 msgr2=0x7fb8b815dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 shutdown_connections 2024-04-27T07:50:33.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.158+0000 7fb8bee1c640 1 -- 172.21.15.183:0/617139301 wait complete. 2024-04-27T07:50:33.175 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new ea39fb67-4673-40e4-b2b1-8866590cdef3 6 2024-04-27T07:50:33.186 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.182+0000 7f401ddd4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T07:50:33.189 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.185+0000 7fa3f1740180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T07:50:33.266 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.262+0000 7f401ddd4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T07:50:33.269 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.265+0000 7fa3f1740180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T07:50:33.298 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.294+0000 7f0d7fc11640 1 Processor -- start 2024-04-27T07:50:33.299 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.294+0000 7f0d7fc11640 1 -- start start 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7fc11640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d78160c10 0x7f0d780d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7fc11640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780cdc90 con 0x7f0d7815ffa0 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7fc11640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780cdf70 con 0x7f0d780b3440 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7fc11640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780d0150 con 0x7f0d78160c10 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d1ac640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0d780b3440 0x7f0d78160500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:39734/0 (socket says 172.21.15.183:39734) 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d1ac640 1 -- 172.21.15.183:0/3954868127 learned_addr learned my addr 172.21.15.183:0/3954868127 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7e1ae640 1 --2- 172.21.15.183:0/3954868127 >> v2:172.21.15.22:3300/0 conn(0x7f0d78160c10 0x7f0d780d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756113510 0 0) 0x7f0d780cdf70 con 0x7f0d780b3440 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d64003530 con 0x7f0d780b3440 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2891532663 0 0) 0x7f0d64003530 con 0x7f0d780b3440 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 >> v2:172.21.15.22:3300/0 conn(0x7f0d78160c10 msgr2=0x7f0d780d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:33.302 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 --2- 172.21.15.183:0/3954868127 >> v2:172.21.15.22:3300/0 conn(0x7f0d78160c10 0x7f0d780d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 >> v1:172.21.15.22:6789/0 conn(0x7f0d7815ffa0 legacy=0x7f0d780b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780ce250 con 0x7f0d780b3440 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0d68001680 con 0x7f0d780b3440 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 >> v1:172.21.15.183:6789/0 conn(0x7f0d780b3440 legacy=0x7f0d78160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 0x7f0d64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d64004810 0x7f0d64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d640051d0 0x7f0d6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780d0150 con 0x7f0d64004810 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6400c170 con 0x7f0d640039c0 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6400c370 con 0x7f0d640051d0 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 0x7f0d64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7e1ae640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d640051d0 0x7f0d6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d1ac640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d64004810 0x7f0d64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 -- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d640051d0 msgr2=0x7f0d6400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:33.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d640051d0 0x7f0d6400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.304 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 -- 172.21.15.183:0/3954868127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d64004810 msgr2=0x7f0d64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:33.304 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d64004810 0x7f0d64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.304 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 -- 172.21.15.183:0/3954868127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d64003530 con 0x7f0d640039c0 2024-04-27T07:50:33.304 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7e1ae640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d640051d0 0x7f0d6400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:33.304 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7e1ae640 1 -- 172.21.15.183:0/3954868127 reap_dead start 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.298+0000 7f0d7d9ad640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 0x7f0d64003de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001780 tx=0x7f0d7401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30a566fcb3bb557 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740013a0 con 0x7f0d640039c0 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d740052f0 con 0x7f0d640039c0 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7c9ab640 1 -- 172.21.15.183:0/3954868127 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003840 con 0x7f0d640039c0 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 msgr2=0x7f0d64003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 0x7f0d64003de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001780 tx=0x7f0d7401c400 comp rx=0 tx=0).stop 2024-04-27T07:50:33.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- 172.21.15.183:0/3954868127 shutdown_connections 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/3954868127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d640039c0 0x7f0d64003de0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- 172.21.15.183:0/3954868127 >> 172.21.15.183:0/3954868127 conn(0x7f0d780b4070 msgr2=0x7f0d780b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- 172.21.15.183:0/3954868127 shutdown_connections 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- 172.21.15.183:0/3954868127 wait complete. 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 Processor -- start 2024-04-27T07:50:33.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.302+0000 7f0d7fc11640 1 -- start start 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d640039c0 0x7f0d780b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 0x7f0d780b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78161360 con 0x7f0d640039c0 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780d10d0 con 0x7f0d780c9650 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0d780d1250 con 0x7f0d780b58a0 2024-04-27T07:50:33.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7e1ae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 0x7f0d780b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38598/0 (socket says 172.21.15.183:38598) 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 -- 172.21.15.183:0/2258004614 learned_addr learned my addr 172.21.15.183:0/2258004614 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d9ad640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d640039c0 0x7f0d780b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 -- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 msgr2=0x7f0d780b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 0x7f0d780b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 -- 172.21.15.183:0/2258004614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d640039c0 msgr2=0x7f0d780b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d640039c0 0x7f0d780b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 -- 172.21.15.183:0/2258004614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780cafc0 con 0x7f0d780c9650 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7e1ae640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 0x7f0d780b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7d1ac640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d680072c0 tx=0x7f0d68007780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800f9c0 con 0x7f0d780c9650 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68018070 con 0x7f0d780c9650 2024-04-27T07:50:33.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d780b6ba0 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800fdc0 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d780b7160 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d6801d460 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f0d68013d00 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.306+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7804ff10 con 0x7f0d780c9650 2024-04-27T07:50:33.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.310+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0d68013710 con 0x7f0d780c9650 2024-04-27T07:50:33.324 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.322+0000 7f401ddd4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T07:50:33.325 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.321+0000 7fa3f1740180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T07:50:33.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.378+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea39fb67-4673-40e4-b2b1-8866590cdef3", "id": 6} v 0) -- 0x7f0d780b4cd0 con 0x7f0d780c9650 2024-04-27T07:50:33.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d6e7fc640 1 -- 172.21.15.183:0/2258004614 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea39fb67-4673-40e4-b2b1-8866590cdef3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0d680138f0 con 0x7f0d780c9650 2024-04-27T07:50:33.385 INFO:teuthology.orchestra.run.smithi183.stdout:6 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 msgr2=0x7f0d780c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d680072c0 tx=0x7f0d68007780 comp rx=0 tx=0).stop 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 shutdown_connections 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d780b58a0 0x7f0d780b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d780c9650 0x7f0d780c9ad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 --2- 172.21.15.183:0/2258004614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d640039c0 0x7f0d780b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 >> 172.21.15.183:0/2258004614 conn(0x7f0d780b4070 msgr2=0x7f0d7815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 shutdown_connections 2024-04-27T07:50:33.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.382+0000 7f0d7fc11640 1 -- 172.21.15.183:0/2258004614 wait complete. 2024-04-27T07:50:33.403 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 9f9f8998-7b99-4dd6-9637-f1b273f62fa2 7 2024-04-27T07:50:33.480 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.478+0000 7f401ddd4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T07:50:33.482 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.481+0000 7fa3f1740180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T07:50:33.521 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 Processor -- start 2024-04-27T07:50:33.521 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 -- start start 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5015d410 con 0x7f1e5006f0b0 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5015ccc0 con 0x7f1e5015b5e0 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e55971640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5015e020 con 0x7f1e500cfed0 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38610/0 (socket says 172.21.15.183:38610) 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4effd640 1 -- 172.21.15.183:0/1605858055 learned_addr learned my addr 172.21.15.183:0/1605858055 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:35594/0 (socket says 172.21.15.183:35594) 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 -- 172.21.15.183:0/1605858055 >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 msgr2=0x7f1e5015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 --2- 172.21.15.183:0/1605858055 >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 -- 172.21.15.183:0/1605858055 >> v1:172.21.15.22:6789/0 conn(0x7f1e5006f0b0 legacy=0x7f1e500cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4f7fe640 1 -- 172.21.15.183:0/1605858055 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5015d6f0 con 0x7f1e500cfed0 2024-04-27T07:50:33.524 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.518+0000 7f1e4effd640 1 --2- 172.21.15.183:0/1605858055 >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:33.525 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e4f7fe640 1 --2- 172.21.15.183:0/1605858055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bddf97a04fda5d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.525 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e4dffb640 1 -- 172.21.15.183:0/1605858055 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400a7070 con 0x7f1e500cfed0 2024-04-27T07:50:33.525 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e4dffb640 1 -- 172.21.15.183:0/1605858055 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e400b0070 con 0x7f1e500cfed0 2024-04-27T07:50:33.525 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e4dffb640 1 -- 172.21.15.183:0/1605858055 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ab890 con 0x7f1e500cfed0 2024-04-27T07:50:33.526 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 -- 172.21.15.183:0/1605858055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 msgr2=0x7f1e500ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.526 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 --2- 172.21.15.183:0/1605858055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c470 comp rx=0 tx=0).stop 2024-04-27T07:50:33.526 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 -- 172.21.15.183:0/1605858055 shutdown_connections 2024-04-27T07:50:33.526 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 --2- 172.21.15.183:0/1605858055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e500ce050 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.526 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 --2- 172.21.15.183:0/1605858055 >> v2:172.21.15.183:3300/0 conn(0x7f1e5015b5e0 0x7f1e5015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 -- 172.21.15.183:0/1605858055 >> 172.21.15.183:0/1605858055 conn(0x7f1e5009aa30 msgr2=0x7f1e500d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 -- 172.21.15.183:0/1605858055 shutdown_connections 2024-04-27T07:50:33.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.522+0000 7f1e55971640 1 -- 172.21.15.183:0/1605858055 wait complete. 2024-04-27T07:50:33.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 Processor -- start 2024-04-27T07:50:33.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- start start 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e5006f0b0 0x7f1e501532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e501f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5015d1f0 con 0x7f1e5006f0b0 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5015d070 con 0x7f1e5015b5e0 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e500d1df0 con 0x7f1e500cfed0 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e501f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38620/0 (socket says 172.21.15.183:38620) 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4f7fe640 1 -- 172.21.15.183:0/531108126 learned_addr learned my addr 172.21.15.183:0/531108126 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4effd640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e5006f0b0 0x7f1e501532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 -- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 msgr2=0x7f1e501f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 -- 172.21.15.183:0/531108126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e5006f0b0 msgr2=0x7f1e501532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e5006f0b0 0x7f1e501532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 -- 172.21.15.183:0/531108126 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e500ca7b0 con 0x7f1e500cfed0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4f7fe640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e4e7fc640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e501f6ef0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c3f0 tx=0x7f1e4400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440052d0 con 0x7f1e500cfed0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e44005470 con 0x7f1e500cfed0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e500caa90 con 0x7f1e500cfed0 2024-04-27T07:50:33.531 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44007260 con 0x7f1e500cfed0 2024-04-27T07:50:33.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e500c6150 con 0x7f1e500cfed0 2024-04-27T07:50:33.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e44007400 con 0x7f1e500cfed0 2024-04-27T07:50:33.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f1e44011bf0 con 0x7f1e500cfed0 2024-04-27T07:50:33.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.526+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5004ff10 con 0x7f1e500cfed0 2024-04-27T07:50:33.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.530+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1e440039c0 con 0x7f1e500cfed0 2024-04-27T07:50:33.544 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:33.542+0000 7f401ddd4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T07:50:33.546 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:33.545+0000 7fa3f1740180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T07:50:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.598+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9f9f8998-7b99-4dd6-9637-f1b273f62fa2", "id": 7} v 0) -- 0x7f1e500cec60 con 0x7f1e500cfed0 2024-04-27T07:50:33.609 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e37fff640 1 -- 172.21.15.183:0/531108126 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f9f8998-7b99-4dd6-9637-f1b273f62fa2", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f1e44003ba0 con 0x7f1e500cfed0 2024-04-27T07:50:33.609 INFO:teuthology.orchestra.run.smithi183.stdout:7 2024-04-27T07:50:33.610 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 msgr2=0x7f1e501f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.610 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e501f6ef0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c3f0 tx=0x7f1e4400c8b0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.610 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 shutdown_connections 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5015b5e0 0x7f1e501f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e500cfed0 0x7f1e501f6ef0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 --2- 172.21.15.183:0/531108126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e5006f0b0 0x7f1e501532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 >> 172.21.15.183:0/531108126 conn(0x7f1e5009aa30 msgr2=0x7f1e500cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 shutdown_connections 2024-04-27T07:50:33.611 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:50:33.606+0000 7f1e55971640 1 -- 172.21.15.183:0/531108126 wait complete. 2024-04-27T07:50:33.625 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-27T07:50:33.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-27T07:50:33.628 INFO:tasks.ceph.osd.0:Started 2024-04-27T07:50:33.628 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-27T07:50:33.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-27T07:50:33.632 INFO:tasks.ceph.osd.1:Started 2024-04-27T07:50:33.632 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-27T07:50:33.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-27T07:50:33.634 INFO:tasks.ceph.osd.2:Started 2024-04-27T07:50:33.634 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-27T07:50:33.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-27T07:50:33.637 INFO:tasks.ceph.osd.3:Started 2024-04-27T07:50:33.637 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-27T07:50:33.637 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-27T07:50:33.639 INFO:tasks.ceph.osd.4:Started 2024-04-27T07:50:33.639 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-27T07:50:33.639 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-27T07:50:33.642 INFO:tasks.ceph.osd.5:Started 2024-04-27T07:50:33.642 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-27T07:50:33.642 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-27T07:50:33.645 INFO:tasks.ceph.osd.6:Started 2024-04-27T07:50:33.645 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-27T07:50:33.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-27T07:50:33.647 INFO:tasks.ceph.osd.7:Started 2024-04-27T07:50:33.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 Processor -- start 2024-04-27T07:50:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 -- start start 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fafc015eba0 0x7fafc015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 0x7fafc0168620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc006e940 con 0x7fafc00b3e00 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fafc015f650 con 0x7fafc015eba0 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc8764640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fafc015f930 con 0x7fafc006d2f0 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc5cff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fafc00b3e00 0x7fafc006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35106/0 (socket says 172.21.15.22:35106) 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc5cff640 1 -- 172.21.15.22:0/4250173150 learned_addr learned my addr 172.21.15.22:0/4250173150 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6500640 1 --2- 172.21.15.22:0/4250173150 >> v2:172.21.15.22:3300/0 conn(0x7fafc015eba0 0x7fafc015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6d01640 1 --2- 172.21.15.22:0/4250173150 >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 0x7fafc0168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc54fe640 1 -- 172.21.15.22:0/4250173150 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210028835 0 0) 0x7fafc006e940 con 0x7fafc00b3e00 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc54fe640 1 -- 172.21.15.22:0/4250173150 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa8003530 con 0x7fafc00b3e00 2024-04-27T07:50:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6500640 1 -- 172.21.15.22:0/4250173150 >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 msgr2=0x7fafc0168620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6500640 1 --2- 172.21.15.22:0/4250173150 >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 0x7fafc0168620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6500640 1 -- 172.21.15.22:0/4250173150 >> v1:172.21.15.22:6789/0 conn(0x7fafc00b3e00 legacy=0x7fafc006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6500640 1 -- 172.21.15.22:0/4250173150 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc015fc10 con 0x7fafc015eba0 2024-04-27T07:50:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.769+0000 7fafc6d01640 1 --2- 172.21.15.22:0/4250173150 >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 0x7fafc0168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6500640 1 --2- 172.21.15.22:0/4250173150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc015ef80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009d90 tx=0x7fafb001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8202e41e0f939b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc54fe640 1 -- 172.21.15.22:0/4250173150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00b2070 con 0x7fafc015eba0 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc54fe640 1 -- 172.21.15.22:0/4250173150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb00a59b0 con 0x7fafc015eba0 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc54fe640 1 -- 172.21.15.22:0/4250173150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00a5d80 con 0x7fafc015eba0 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/4250173150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 msgr2=0x7fafc015ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- 172.21.15.22:0/4250173150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc015ef80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009d90 tx=0x7fafb001c8d0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/4250173150 shutdown_connections 2024-04-27T07:50:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- 172.21.15.22:0/4250173150 >> v2:172.21.15.183:3300/0 conn(0x7fafc006d2f0 0x7fafc0168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- 172.21.15.22:0/4250173150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc015ef80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/4250173150 >> 172.21.15.22:0/4250173150 conn(0x7fafc0072870 msgr2=0x7fafc0072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/4250173150 shutdown_connections 2024-04-27T07:50:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/4250173150 wait complete. 2024-04-27T07:50:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 Processor -- start 2024-04-27T07:50:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- start start 2024-04-27T07:50:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc006d2f0 0x7fafc014a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafc00b3e00 0x7fafc015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fafc016b710 con 0x7fafc015eba0 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafc016b590 con 0x7fafc006d2f0 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fafc016b890 con 0x7fafc00b3e00 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44668/0 (socket says 172.21.15.22:44668) 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 -- 172.21.15.22:0/2415062925 learned_addr learned my addr 172.21.15.22:0/2415062925 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6500640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc006d2f0 0x7fafc014a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc5cff640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafc00b3e00 0x7fafc015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 -- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafc00b3e00 msgr2=0x7fafc015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafc00b3e00 0x7fafc015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 -- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc006d2f0 msgr2=0x7fafc014a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc006d2f0 0x7fafc014a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 -- 172.21.15.22:0/2415062925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc015e340 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc6d01640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e630 tx=0x7fafb800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8017070 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb800c620 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb800f040 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/2415062925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc01524e0 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.773+0000 7fafc8764640 1 -- 172.21.15.22:0/2415062925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc00b2500 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.777+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fafb801a070 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.777+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fafb801cbd0 con 0x7fafc015eba0 2024-04-27T07:50:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.777+0000 7fafc8764640 1 -- 172.21.15.22:0/2415062925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0069340 con 0x7fafc015eba0 2024-04-27T07:50:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.777+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fafb801c450 con 0x7fafc015eba0 2024-04-27T07:50:33.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.837+0000 7fafc8764640 1 -- 172.21.15.22:0/2415062925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafc0154a90 con 0x7fafc015eba0 2024-04-27T07:50:33.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.841+0000 7fafbeffd640 1 -- 172.21.15.22:0/2415062925 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fafb801c630 con 0x7fafc015eba0 2024-04-27T07:50:33.843 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:33.843 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:33.601972+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 -- 172.21.15.22:0/2415062925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 msgr2=0x7fafc014b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e630 tx=0x7fafb800eaf0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 -- 172.21.15.22:0/2415062925 shutdown_connections 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafc015eba0 0x7fafc014b000 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafc00b3e00 0x7fafc015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 --2- 172.21.15.22:0/2415062925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc006d2f0 0x7fafc014a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 -- 172.21.15.22:0/2415062925 >> 172.21.15.22:0/2415062925 conn(0x7fafc0072870 msgr2=0x7fafc00adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:33.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 -- 172.21.15.22:0/2415062925 shutdown_connections 2024-04-27T07:50:33.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:33.845+0000 7fafbcff9640 1 -- 172.21.15.22:0/2415062925 wait complete. 2024-04-27T07:50:33.884 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-27T07:50:33.885 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-27T07:50:34.153 INFO:tasks.ceph.mgr.y.smithi022.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-27T07:50:34.153 INFO:tasks.ceph.mgr.y.smithi022.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-27T07:50:34.153 INFO:tasks.ceph.mgr.y.smithi022.stderr: from numpy import show_config as show_numpy_config 2024-04-27T07:50:34.211 INFO:tasks.ceph.mgr.x.smithi183.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-27T07:50:34.211 INFO:tasks.ceph.mgr.x.smithi183.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-27T07:50:34.211 INFO:tasks.ceph.mgr.x.smithi183.stderr: from numpy import show_config as show_numpy_config 2024-04-27T07:50:34.213 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:34.209+0000 7fa3f1740180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T07:50:34.245 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-04-27T07:50:34.242+0000 7f03483217c0 -1 Falling back to public interface 2024-04-27T07:50:34.247 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-04-27T07:50:34.245+0000 7f35830747c0 -1 Falling back to public interface 2024-04-27T07:50:34.257 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-04-27T07:50:34.254+0000 7f95c967d7c0 -1 Falling back to public interface 2024-04-27T07:50:34.258 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:34.254+0000 7f401ddd4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T07:50:34.262 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-04-27T07:50:34.261+0000 7ff3e400b7c0 -1 Falling back to public interface 2024-04-27T07:50:34.264 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-04-27T07:50:34.262+0000 7ffa19bc77c0 -1 Falling back to public interface 2024-04-27T07:50:34.272 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-04-27T07:50:34.270+0000 7ff72e3e57c0 -1 Falling back to public interface 2024-04-27T07:50:34.274 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-04-27T07:50:34.273+0000 7fa693b587c0 -1 Falling back to public interface 2024-04-27T07:50:34.275 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:34.273+0000 7f339ec5f7c0 -1 Falling back to public interface 2024-04-27T07:50:34.275 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:34.273+0000 7f339dc1f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-04-27T07:50:34.277 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:34.273+0000 7f339dc1f640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6825/0: (98) Address already in use 2024-04-27T07:50:34.402 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:34.401+0000 7fa3f1740180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:50:34.431 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:34.426+0000 7f401ddd4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:50:34.561 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:34.557+0000 7fa3f1740180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T07:50:34.590 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:34.586+0000 7f401ddd4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T07:50:34.628 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:34.625+0000 7fa3f1740180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T07:50:34.660 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:34.658+0000 7f401ddd4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T07:50:34.690 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:34.770 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:34.769+0000 7fa3f1740180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:50:34.806 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 Processor -- start 2024-04-27T07:50:34.806 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 -- start start 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f66540b2e70 0x7f66540b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f665409b180 0x7f66540d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665406d020 con 0x7f665415feb0 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f665406d300 con 0x7f66540b2e70 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66597d5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f665406d5e0 con 0x7f665409b180 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f66540b2e70 0x7f66540b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f66540b2e70 0x7f66540b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44672/0 (socket says 172.21.15.22:44672) 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 -- 172.21.15.22:0/507478952 learned_addr learned my addr 172.21.15.22:0/507478952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:34.808 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f66537fe640 1 --2- 172.21.15.22:0/507478952 >> v2:172.21.15.183:3300/0 conn(0x7f665409b180 0x7f66540d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:34.809 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 -- 172.21.15.22:0/507478952 >> v2:172.21.15.183:3300/0 conn(0x7f665409b180 msgr2=0x7f66540d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:34.809 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 --2- 172.21.15.22:0/507478952 >> v2:172.21.15.183:3300/0 conn(0x7f665409b180 0x7f66540d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.809 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 -- 172.21.15.22:0/507478952 >> v1:172.21.15.183:6789/0 conn(0x7f665415feb0 legacy=0x7f665409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:34.809 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 -- 172.21.15.22:0/507478952 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665406d8c0 con 0x7f66540b2e70 2024-04-27T07:50:34.810 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:34.802+0000 7f401ddd4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.805+0000 7f6652ffd640 1 --2- 172.21.15.22:0/507478952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66540b2e70 0x7f66540b3380 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6654073420 tx=0x7f664401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f01fc1c5acdae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6651ffb640 1 -- 172.21.15.22:0/507478952 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440b0070 con 0x7f66540b2e70 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6651ffb640 1 -- 172.21.15.22:0/507478952 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66440ab600 con 0x7f66540b2e70 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6651ffb640 1 -- 172.21.15.22:0/507478952 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440ab9d0 con 0x7f66540b2e70 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/507478952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66540b2e70 msgr2=0x7f66540b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- 172.21.15.22:0/507478952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66540b2e70 0x7f66540b3380 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6654073420 tx=0x7f664401c490 comp rx=0 tx=0).stop 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/507478952 shutdown_connections 2024-04-27T07:50:34.811 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- 172.21.15.22:0/507478952 >> v2:172.21.15.183:3300/0 conn(0x7f665409b180 0x7f66540d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.812 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- 172.21.15.22:0/507478952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66540b2e70 0x7f66540b3380 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.812 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/507478952 >> 172.21.15.22:0/507478952 conn(0x7f66540b40e0 msgr2=0x7f66540b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:34.812 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/507478952 shutdown_connections 2024-04-27T07:50:34.812 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/507478952 wait complete. 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 Processor -- start 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- start start 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540b2e70 0x7f6654158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 0x7f6654158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66540d17f0 con 0x7f665409b180 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66540d1670 con 0x7f665415feb0 2024-04-27T07:50:34.815 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f66540d1970 con 0x7f66540b2e70 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44674/0 (socket says 172.21.15.22:44674) 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 -- 172.21.15.22:0/3172057946 learned_addr learned my addr 172.21.15.22:0/3172057946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66537fe640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 0x7f6654158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66527fc640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540b2e70 0x7f6654158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 -- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540b2e70 msgr2=0x7f6654158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540b2e70 0x7f6654158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 -- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 msgr2=0x7f6654158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 0x7f6654158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 -- 172.21.15.22:0/3172057946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66540c7480 con 0x7f665409b180 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66537fe640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 0x7f6654158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f6652ffd640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f66440ae040 tx=0x7f6644019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:34.816 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440b0040 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66440b9070 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440b4660 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66540c6120 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66540c66c0 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66440b4800 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.809+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f66440bd3c0 con 0x7f665409b180 2024-04-27T07:50:34.817 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.813+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665404ff10 con 0x7f665409b180 2024-04-27T07:50:34.818 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.817+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6644016110 con 0x7f665409b180 2024-04-27T07:50:34.876 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.873+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f665415bc00 con 0x7f665409b180 2024-04-27T07:50:34.877 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.873+0000 7f663b7fe640 1 -- 172.21.15.22:0/3172057946 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f66440b4be0 con 0x7f665409b180 2024-04-27T07:50:34.877 INFO:teuthology.misc.health.smithi022.stdout: 2024-04-27T07:50:34.877 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":10,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:33.601972+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:34.878 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 msgr2=0x7f6654156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:34.878 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f66440ae040 tx=0x7f6644019230 comp rx=0 tx=0).stop 2024-04-27T07:50:34.878 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 shutdown_connections 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415feb0 0x7f6654158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540b2e70 0x7f6654158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 --2- 172.21.15.22:0/3172057946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f665409b180 0x7f6654156ee0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 >> 172.21.15.22:0/3172057946 conn(0x7f66540b40e0 msgr2=0x7f665415c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 shutdown_connections 2024-04-27T07:50:34.879 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:34.877+0000 7f66597d5640 1 -- 172.21.15.22:0/3172057946 wait complete. 2024-04-27T07:50:34.893 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-27T07:50:35.428 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:35.425+0000 7fa3f1740180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T07:50:35.431 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:35.426+0000 7f401ddd4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T07:50:35.507 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:35.505+0000 7fa3f1740180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T07:50:35.508 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:35.506+0000 7f401ddd4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T07:50:35.690 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:35.686+0000 7f401ddd4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T07:50:35.696 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:35.693+0000 7fa3f1740180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T07:50:35.763 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:35.758+0000 7f401ddd4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T07:50:35.773 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:35.769+0000 7fa3f1740180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T07:50:36.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:36.050+0000 7f401ddd4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T07:50:36.080 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:36.077+0000 7fa3f1740180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T07:50:36.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:36.110+0000 7f401ddd4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T07:50:36.146 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-04-27T07:50:36.145+0000 7fa3f1740180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T07:50:37.154 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:37.150+0000 7f3fc5221640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-27T07:50:37.156 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-04-27T07:50:37.154+0000 7f3fc5221640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-27T07:50:38.064 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-04-27T07:50:38.062+0000 7ffa19bc77c0 -1 osd.5 0 log_to_monitors true 2024-04-27T07:50:38.105 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-04-27T07:50:38.100+0000 7fa693b587c0 -1 osd.3 0 log_to_monitors true 2024-04-27T07:50:38.114 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-04-27T07:50:38.112+0000 7f35830747c0 -1 osd.0 0 log_to_monitors true 2024-04-27T07:50:38.123 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-04-27T07:50:38.118+0000 7f03483217c0 -1 osd.4 0 log_to_monitors true 2024-04-27T07:50:38.135 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-04-27T07:50:38.132+0000 7ff3e400b7c0 -1 osd.2 0 log_to_monitors true 2024-04-27T07:50:38.139 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:38.136+0000 7f339ec5f7c0 -1 osd.1 0 log_to_monitors true 2024-04-27T07:50:38.235 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-04-27T07:50:38.234+0000 7f95c967d7c0 -1 osd.6 0 log_to_monitors true 2024-04-27T07:50:38.247 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-04-27T07:50:38.242+0000 7ff72e3e57c0 -1 osd.7 0 log_to_monitors true 2024-04-27T07:50:40.638 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-04-27T07:50:40.632+0000 7fa68fb01640 -1 osd.3 0 waiting for initial osdmap 2024-04-27T07:50:40.656 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-04-27T07:50:40.654+0000 7ffa15b70640 -1 osd.5 0 waiting for initial osdmap 2024-04-27T07:50:40.662 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-04-27T07:50:40.660+0000 7fa687935640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:40.714 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-04-27T07:50:40.710+0000 7ffa0d9a4640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:40.721 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-04-27T07:50:40.716+0000 7f357f01d640 -1 osd.0 0 waiting for initial osdmap 2024-04-27T07:50:40.733 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-04-27T07:50:40.730+0000 7f0344adc640 -1 osd.4 0 waiting for initial osdmap 2024-04-27T07:50:40.743 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-04-27T07:50:40.740+0000 7f3576e51640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:40.770 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:40.768+0000 7f339ac08640 -1 osd.1 0 waiting for initial osdmap 2024-04-27T07:50:40.779 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-04-27T07:50:40.776+0000 7ff3e07c6640 -1 osd.2 0 waiting for initial osdmap 2024-04-27T07:50:40.797 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-04-27T07:50:40.794+0000 7f033c0fe640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:40.806 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-04-27T07:50:40.800+0000 7f3392a3c640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:40.810 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-04-27T07:50:40.808+0000 7ff3d7de8640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:41.511 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-04-27T07:50:41.506+0000 7ff72a38e640 -1 osd.7 0 waiting for initial osdmap 2024-04-27T07:50:41.536 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-04-27T07:50:41.533+0000 7ff7221c2640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:41.564 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-04-27T07:50:41.561+0000 7f95c5626640 -1 osd.6 0 waiting for initial osdmap 2024-04-27T07:50:41.597 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-04-27T07:50:41.593+0000 7f95bd45a640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:50:41.699 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:41.823 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 Processor -- start 2024-04-27T07:50:41.823 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 -- start start 2024-04-27T07:50:41.824 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe02c15b900 0x7fe02c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.824 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 0x7fe02c165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.824 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02c0cc1d0 con 0x7fe02c0cf610 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe02c0cbfa0 con 0x7fe02c15b900 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe030b68640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe02c0701a0 con 0x7fe02c0ce470 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe02c15b900 0x7fe02c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02ad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 0x7fe02c165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe02c15b900 0x7fe02c0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:59578/0 (socket says 172.21.15.22:59578) 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe029d9b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe02c0cf610 0x7fe02c0cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38420/0 (socket says 172.21.15.22:38420) 2024-04-27T07:50:41.825 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 -- 172.21.15.22:0/594061339 learned_addr learned my addr 172.21.15.22:0/594061339 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:41.826 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 -- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 msgr2=0x7fe02c165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:41.826 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 0x7fe02c165330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:41.826 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 -- 172.21.15.22:0/594061339 >> v1:172.21.15.183:6789/0 conn(0x7fe02c0cf610 legacy=0x7fe02c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:41.826 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02a59c640 1 -- 172.21.15.22:0/594061339 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c0cc4b0 con 0x7fe02c15b900 2024-04-27T07:50:41.826 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.820+0000 7fe02ad9d640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 0x7fe02c165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:41.827 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe02a59c640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c15b900 0x7fe02c0cf0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe024009e50 tx=0x7fe02401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557499bcde6185e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:41.829 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe02959a640 1 -- 172.21.15.22:0/594061339 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240a5900 con 0x7fe02c15b900 2024-04-27T07:50:41.829 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe02959a640 1 -- 172.21.15.22:0/594061339 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0240a5aa0 con 0x7fe02c15b900 2024-04-27T07:50:41.829 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe02959a640 1 -- 172.21.15.22:0/594061339 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240ad3f0 con 0x7fe02c15b900 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 -- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c15b900 msgr2=0x7fe02c0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c15b900 0x7fe02c0cf0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe024009e50 tx=0x7fe02401ca30 comp rx=0 tx=0).stop 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 -- 172.21.15.22:0/594061339 shutdown_connections 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c0ce470 0x7fe02c165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 --2- 172.21.15.22:0/594061339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c15b900 0x7fe02c0cf0d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 -- 172.21.15.22:0/594061339 >> 172.21.15.22:0/594061339 conn(0x7fe02c09aa70 msgr2=0x7fe02c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 -- 172.21.15.22:0/594061339 shutdown_connections 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 -- 172.21.15.22:0/594061339 wait complete. 2024-04-27T07:50:41.832 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.824+0000 7fe030b68640 1 Processor -- start 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- start start 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c0cf610 0x7fe02c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c15b900 0x7fe02c152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c0cd270 con 0x7fe02c0ce470 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c0b3a10 con 0x7fe02c0cf610 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe02c0b01d0 con 0x7fe02c15b900 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33438/0 (socket says 172.21.15.22:33438) 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 -- 172.21.15.22:0/3561086159 learned_addr learned my addr 172.21.15.22:0/3561086159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02ad9d640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c15b900 0x7fe02c152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe029d9b640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c0cf610 0x7fe02c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 -- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c15b900 msgr2=0x7fe02c152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c15b900 0x7fe02c152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:41.833 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 -- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c0cf610 msgr2=0x7fe02c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c0cf610 0x7fe02c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 -- 172.21.15.22:0/3561086159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c0af8e0 con 0x7fe02c0ce470 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe02a59c640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe02401a660 tx=0x7fe0240197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240b8470 con 0x7fe02c0ce470 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0240b8630 con 0x7fe02c0ce470 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240b8a20 con 0x7fe02c0ce470 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02c0afb80 con 0x7fe02c0ce470 2024-04-27T07:50:41.834 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02c1647d0 con 0x7fe02c0ce470 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02c04ff10 con 0x7fe02c0ce470 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fe0240c0020 con 0x7fe02c0ce470 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.828+0000 7fe01affd640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 0x7fe00c06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.832+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7fe024120c90 con 0x7fe02c0ce470 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.832+0000 7fe029d9b640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 0x7fe00c06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:41.835 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.832+0000 7fe029d9b640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 0x7fe00c06a7b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe02c1654c0 tx=0x7fe020007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:41.838 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.836+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe024126020 con 0x7fe02c0ce470 2024-04-27T07:50:41.997 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.992+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe02c0cb8c0 con 0x7fe02c0ce470 2024-04-27T07:50:41.998 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe01affd640 1 -- 172.21.15.22:0/3561086159 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10885 (secure 0 0 0) 0x7fe0240c7020 con 0x7fe02c0ce470 2024-04-27T07:50:41.998 INFO:teuthology.misc.health.smithi022.stdout: 2024-04-27T07:50:41.999 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":14,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:41.663816+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 msgr2=0x7fe00c06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 0x7fe00c06a7b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe02c1654c0 tx=0x7fe020007cb0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 msgr2=0x7fe02c164060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe02401a660 tx=0x7fe0240197d0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 shutdown_connections 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe00c0682f0 0x7fe00c06a7b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe02c15b900 0x7fe02c152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c0cf610 0x7fe02c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 --2- 172.21.15.22:0/3561086159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe02c0ce470 0x7fe02c164060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.001 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 >> 172.21.15.22:0/3561086159 conn(0x7fe02c09aa70 msgr2=0x7fe02c0b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:42.002 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 shutdown_connections 2024-04-27T07:50:42.002 INFO:teuthology.misc.health.smithi022.stderr:2024-04-27T07:50:41.996+0000 7fe030b68640 1 -- 172.21.15.22:0/3561086159 wait complete. 2024-04-27T07:50:42.015 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-27T07:50:42.015 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-27T07:50:42.016 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-27T07:50:42.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-27T07:50:42.018 INFO:tasks.ceph.mds.a:Started 2024-04-27T07:50:42.018 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-27T07:50:42.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-27T07:50:42.020 INFO:tasks.ceph.mds.c:Started 2024-04-27T07:50:42.020 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-27T07:50:42.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-27T07:50:42.023 INFO:tasks.ceph.mds.b:Started 2024-04-27T07:50:42.023 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-27T07:50:42.023 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-27T07:50:42.026 INFO:tasks.ceph.mds.d:Started 2024-04-27T07:50:42.026 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-27T07:50:42.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:42.093 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-04-27T07:50:42.097 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-04-27T07:50:42.106 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2024-04-27T07:50:42.106 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2024-04-27T07:50:42.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 Processor -- start 2024-04-27T07:50:42.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 -- start start 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa4f40cdfe0 0x7fa4f40b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40ce470 con 0x7fa4f40d05a0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f415cb50 con 0x7fa4f40d0af0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fc371640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f409bc70 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4f990c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa4f40d05a0 0x7fa4f409b3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49756/0 (socket says 172.21.15.22:49756) 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4f990c640 1 -- 172.21.15.22:0/2320382432 learned_addr learned my addr 172.21.15.22:0/2320382432 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fa90e640 1 --2- 172.21.15.22:0/2320382432 >> v2:172.21.15.22:3300/0 conn(0x7fa4f40cdfe0 0x7fa4f40b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158013507 0 0) 0x7fa4e4001120 con 0x7fa4f40d05a0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4e0003530 con 0x7fa4f40d05a0 2024-04-27T07:50:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fa90e640 1 -- 172.21.15.22:0/2320382432 >> v1:172.21.15.183:6789/0 conn(0x7fa4f40d0af0 legacy=0x7fa4f40d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937220207 0 0) 0x7fa4f415cb50 con 0x7fa4f40d0af0 2024-04-27T07:50:42.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fa90e640 1 -- 172.21.15.22:0/2320382432 >> v1:172.21.15.22:6789/0 conn(0x7fa4f40d05a0 legacy=0x7fa4f409b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.128+0000 7fa4fa90e640 1 -- 172.21.15.22:0/2320382432 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40cd220 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fa90e640 1 --2- 172.21.15.22:0/2320382432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40cdfe0 0x7fa4f40b1ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009d90 tx=0x7fa4ec01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eae5f9419520625 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0b2070 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0a5910 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4f910b640 1 -- 172.21.15.22:0/2320382432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0a5cc0 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- 172.21.15.22:0/2320382432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40cdfe0 msgr2=0x7fa4f40b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 --2- 172.21.15.22:0/2320382432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40cdfe0 0x7fa4f40b1ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009d90 tx=0x7fa4ec01c940 comp rx=0 tx=0).stop 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- 172.21.15.22:0/2320382432 shutdown_connections 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 --2- 172.21.15.22:0/2320382432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40cdfe0 0x7fa4f40b1ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- 172.21.15.22:0/2320382432 >> 172.21.15.22:0/2320382432 conn(0x7fa4f40b4080 msgr2=0x7fa4f40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- 172.21.15.22:0/2320382432 shutdown_connections 2024-04-27T07:50:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- 172.21.15.22:0/2320382432 wait complete. 2024-04-27T07:50:42.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 Processor -- start 2024-04-27T07:50:42.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.132+0000 7fa4fc371640 1 -- start start 2024-04-27T07:50:42.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4f40cdfe0 0x7fa4f4150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 0x7fa4f4166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f416a7b0 con 0x7fa4f40d0af0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f416a630 con 0x7fa4f40d05a0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f416a930 con 0x7fa4f40cdfe0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33470/0 (socket says 172.21.15.22:33470) 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 -- 172.21.15.22:0/377808933 learned_addr learned my addr 172.21.15.22:0/377808933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4f990c640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 0x7fa4f4166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa10d640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4f40cdfe0 0x7fa4f4150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 -- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4f40cdfe0 msgr2=0x7fa4f4150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4f40cdfe0 0x7fa4f4150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 -- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 msgr2=0x7fa4f4166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 0x7fa4f4166b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 -- 172.21.15.22:0/377808933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f41675d0 con 0x7fa4f40d0af0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4f990c640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 0x7fa4f4166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa90e640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0b0040 tx=0x7fa4ec0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0bd020 con 0x7fa4f40d0af0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0b8670 con 0x7fa4f40d0af0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f41678b0 con 0x7fa4f40d0af0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0b2040 con 0x7fa4f40d0af0 2024-04-27T07:50:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f4167e70 con 0x7fa4f40d0af0 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4069340 con 0x7fa4f40d0af0 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fa4ec0c0020 con 0x7fa4f40d0af0 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 0x7fa4d806a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7fa4ec1216a0 con 0x7fa4f40d0af0 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa10d640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 0x7fa4d806a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.136+0000 7fa4fa10d640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 0x7fa4d806a7b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0007fd0 tx=0x7fa4f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.148+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa4ec0c7030 con 0x7fa4f40d0af0 2024-04-27T07:50:42.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.308+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f40b8410 con 0x7fa4f40d0af0 2024-04-27T07:50:42.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4eaffd640 1 -- 172.21.15.22:0/377808933 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10885 (secure 0 0 0) 0x7fa4ec016560 con 0x7fa4f40d0af0 2024-04-27T07:50:42.316 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:42.316 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":14,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:41.663816+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 msgr2=0x7fa4d806a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 0x7fa4d806a7b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0007fd0 tx=0x7fa4f0009040 comp rx=0 tx=0).stop 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 msgr2=0x7fa4f41590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0b0040 tx=0x7fa4ec0196e0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 shutdown_connections 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa4d80682f0 0x7fa4d806a7b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4f40d0af0 0x7fa4f41590d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4f40d05a0 0x7fa4f4166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 --2- 172.21.15.22:0/377808933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4f40cdfe0 0x7fa4f4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 >> 172.21.15.22:0/377808933 conn(0x7fa4f40b4080 msgr2=0x7fa4f40b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:42.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 shutdown_connections 2024-04-27T07:50:42.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.312+0000 7fa4fc371640 1 -- 172.21.15.22:0/377808933 wait complete. 2024-04-27T07:50:42.333 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-27T07:50:42.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:50:42.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.448+0000 7f935815a640 1 Processor -- start 2024-04-27T07:50:42.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.448+0000 7f935815a640 1 -- start start 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f93500d4790 0x7f93500d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f935015ebe0 0x7f93500cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93500b3d30 con 0x7f93500b7930 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f93500d4540 con 0x7f93500d4790 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93500ba7a0 con 0x7f935015ebe0 2024-04-27T07:50:42.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f93500d4790 0x7f93500d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f93566f7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f93500b7930 0x7f93500b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49766/0 (socket says 172.21.15.22:49766) 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f93566f7640 1 -- 172.21.15.22:0/3632506880 learned_addr learned my addr 172.21.15.22:0/3632506880 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f93500d4790 0x7f93500d4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33482/0 (socket says 172.21.15.22:33482) 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f93556f5640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f935015ebe0 0x7f93500cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 -- 172.21.15.22:0/3632506880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f935015ebe0 msgr2=0x7f93500cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f935015ebe0 0x7f93500cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 -- 172.21.15.22:0/3632506880 >> v1:172.21.15.22:6789/0 conn(0x7f93500b7930 legacy=0x7f93500b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 -- 172.21.15.22:0/3632506880 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93500d50b0 con 0x7f93500d4790 2024-04-27T07:50:42.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9355ef6640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500d4790 0x7f93500d4b70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f934c009d90 tx=0x7f934c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23ee4ced54726ed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9354ef4640 1 -- 172.21.15.22:0/3632506880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0b2070 con 0x7f93500d4790 2024-04-27T07:50:42.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9354ef4640 1 -- 172.21.15.22:0/3632506880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0a57b0 con 0x7f93500d4790 2024-04-27T07:50:42.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f9354ef4640 1 -- 172.21.15.22:0/3632506880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0a5b80 con 0x7f93500d4790 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- 172.21.15.22:0/3632506880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500d4790 msgr2=0x7f93500d4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500d4790 0x7f93500d4b70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f934c009d90 tx=0x7f934c01c8e0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- 172.21.15.22:0/3632506880 shutdown_connections 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f935015ebe0 0x7f93500cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 --2- 172.21.15.22:0/3632506880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500d4790 0x7f93500d4b70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- 172.21.15.22:0/3632506880 >> 172.21.15.22:0/3632506880 conn(0x7f9350067d10 msgr2=0x7f9350068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- 172.21.15.22:0/3632506880 shutdown_connections 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.452+0000 7f935815a640 1 -- 172.21.15.22:0/3632506880 wait complete. 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 Processor -- start 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- start start 2024-04-27T07:50:42.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93500b7930 0x7f93500cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93500d4790 0x7f93500c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93500ba2a0 con 0x7f93500c8590 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93500ba120 con 0x7f93500b7930 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93500cfda0 con 0x7f93500d4790 2024-04-27T07:50:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33488/0 (socket says 172.21.15.22:33488) 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 -- 172.21.15.22:0/3040092981 learned_addr learned my addr 172.21.15.22:0/3040092981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f9355ef6640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93500b7930 0x7f93500cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93556f5640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93500d4790 0x7f93500c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 -- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93500d4790 msgr2=0x7f93500c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93500d4790 0x7f93500c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 -- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93500b7930 msgr2=0x7f93500cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93500b7930 0x7f93500cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 -- 172.21.15.22:0/3040092981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93500ce710 con 0x7f93500c8590 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93566f7640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f934800c400 tx=0x7f934800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348017070 con 0x7f93500c8590 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9348005cf0 con 0x7f93500c8590 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93500c6160 con 0x7f93500c8590 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480072d0 con 0x7f93500c8590 2024-04-27T07:50:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93500ce150 con 0x7f93500c8590 2024-04-27T07:50:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.456+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9350068c30 con 0x7f93500c8590 2024-04-27T07:50:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.460+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f934801a070 con 0x7f93500c8590 2024-04-27T07:50:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.460+0000 7f93467fc640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 0x7f933406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.460+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f9348085610 con 0x7f93500c8590 2024-04-27T07:50:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.460+0000 7f9355ef6640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 0x7f933406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.460+0000 7f9355ef6640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 0x7f933406a7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f934c0af000 tx=0x7f934c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.464+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f93500b3d30 con 0x7f93500c8590 2024-04-27T07:50:42.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.660+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9350153a70 con 0x7f93500c8590 2024-04-27T07:50:42.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.660+0000 7f93467fc640 1 -- 172.21.15.22:0/3040092981 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f9348055b30 con 0x7f93500c8590 2024-04-27T07:50:42.665 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:42.665 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-27T07:50:42.666 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 msgr2=0x7f933406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 0x7f933406a7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f934c0af000 tx=0x7f934c0a7070 comp rx=0 tx=0).stop 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 msgr2=0x7f93500c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f934800c400 tx=0x7f934800c8c0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 shutdown_connections 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9334068320 0x7f933406a7e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93500c8590 0x7f93500c9520 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93500d4790 0x7f93500c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 --2- 172.21.15.22:0/3040092981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93500b7930 0x7f93500cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:42.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 >> 172.21.15.22:0/3040092981 conn(0x7f9350067d10 msgr2=0x7f935015cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:42.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 shutdown_connections 2024-04-27T07:50:42.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:42.664+0000 7f935815a640 1 -- 172.21.15.22:0/3040092981 wait complete. 2024-04-27T07:50:44.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:50:44.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.820+0000 7fd146ab5640 1 Processor -- start 2024-04-27T07:50:44.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.820+0000 7fd146ab5640 1 -- start start 2024-04-27T07:50:44.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14015b870 0x7fd1400b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 0x7fd1400aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd1400aef40 0x7fd1400d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd1400b0b60 con 0x7fd1400aef40 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd1400b4420 con 0x7fd1400b0720 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd146ab5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd1400af710 con 0x7fd14015b870 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd1400aef40 0x7fd1400d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd13ffff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 0x7fd1400aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd13ffff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 0x7fd1400aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:59632/0 (socket says 172.21.15.22:59632) 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd13ffff640 1 -- 172.21.15.22:0/1970874449 learned_addr learned my addr 172.21.15.22:0/1970874449 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd144851640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14015b870 0x7fd1400b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 -- 172.21.15.22:0/1970874449 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14015b870 msgr2=0x7fd1400b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14015b870 0x7fd1400b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 -- 172.21.15.22:0/1970874449 >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 msgr2=0x7fd1400aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 --2- 172.21.15.22:0/1970874449 >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 0x7fd1400aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 -- 172.21.15.22:0/1970874449 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1400af9f0 con 0x7fd1400aef40 2024-04-27T07:50:44.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.824+0000 7fd145052640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd1400d14a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1400727d0 tx=0x7fd13801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faccf9ff6331d5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:44.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd13f7fe640 1 -- 172.21.15.22:0/1970874449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380a8650 con 0x7fd1400aef40 2024-04-27T07:50:44.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd13f7fe640 1 -- 172.21.15.22:0/1970874449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1380a87f0 con 0x7fd1400aef40 2024-04-27T07:50:44.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd13f7fe640 1 -- 172.21.15.22:0/1970874449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380a8bc0 con 0x7fd1400aef40 2024-04-27T07:50:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- 172.21.15.22:0/1970874449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 msgr2=0x7fd1400d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd1400d14a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1400727d0 tx=0x7fd13801c670 comp rx=0 tx=0).stop 2024-04-27T07:50:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- 172.21.15.22:0/1970874449 shutdown_connections 2024-04-27T07:50:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd1400d14a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- 172.21.15.22:0/1970874449 >> v2:172.21.15.183:3300/0 conn(0x7fd1400b0720 0x7fd1400aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- 172.21.15.22:0/1970874449 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14015b870 0x7fd1400b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- 172.21.15.22:0/1970874449 >> 172.21.15.22:0/1970874449 conn(0x7fd1400b6b60 msgr2=0x7fd14009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- 172.21.15.22:0/1970874449 shutdown_connections 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- 172.21.15.22:0/1970874449 wait complete. 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 Processor -- start 2024-04-27T07:50:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- start start 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1400b0720 0x7fd14014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 0x7fd14014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd14006bbd0 con 0x7fd1400aef40 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1400b0d60 con 0x7fd14015b870 2024-04-27T07:50:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd146ab5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd14006bd50 con 0x7fd1400b0720 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33500/0 (socket says 172.21.15.22:33500) 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 -- 172.21.15.22:0/520025383 learned_addr learned my addr 172.21.15.22:0/520025383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd145052640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 0x7fd14014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd13ffff640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1400b0720 0x7fd14014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 -- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1400b0720 msgr2=0x7fd14014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1400b0720 0x7fd14014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 -- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 msgr2=0x7fd14014fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 0x7fd14014fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 -- 172.21.15.22:0/520025383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1400cf2b0 con 0x7fd1400aef40 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd145052640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 0x7fd14014fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.828+0000 7fd144851640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e870 tx=0x7fd13000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130017070 con 0x7fd1400aef40 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13000c940 con 0x7fd1400aef40 2024-04-27T07:50:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1400cf550 con 0x7fd1400aef40 2024-04-27T07:50:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13000ccf0 con 0x7fd1400aef40 2024-04-27T07:50:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1400cfb10 con 0x7fd1400aef40 2024-04-27T07:50:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd140068c30 con 0x7fd1400aef40 2024-04-27T07:50:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fd13001a070 con 0x7fd1400aef40 2024-04-27T07:50:44.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 0x7fd12c06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:44.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd130086ab0 con 0x7fd1400aef40 2024-04-27T07:50:44.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13ffff640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 0x7fd12c06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:44.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.832+0000 7fd13ffff640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 0x7fd12c06a7b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1400b2ef0 tx=0x7fd134008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:44.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:44.836+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd130057610 con 0x7fd1400aef40 2024-04-27T07:50:45.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.036+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd14009acc0 con 0x7fd1400aef40 2024-04-27T07:50:45.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.036+0000 7fd13d7fa640 1 -- 172.21.15.22:0/520025383 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7fd130056f10 con 0x7fd1400aef40 2024-04-27T07:50:45.042 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:45.042 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-27T07:50:45.043 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 msgr2=0x7fd12c06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 0x7fd12c06a7b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1400b2ef0 tx=0x7fd134008040 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 msgr2=0x7fd14014a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e870 tx=0x7fd13000ec70 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 shutdown_connections 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd12c0682f0 0x7fd12c06a7b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd14015b870 0x7fd14014fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1400b0720 0x7fd14014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 --2- 172.21.15.22:0/520025383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1400aef40 0x7fd14014a560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 >> 172.21.15.22:0/520025383 conn(0x7fd1400b6b60 msgr2=0x7fd1400b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 shutdown_connections 2024-04-27T07:50:45.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.040+0000 7fd146ab5640 1 -- 172.21.15.22:0/520025383 wait complete. 2024-04-27T07:50:45.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:45.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 Processor -- start 2024-04-27T07:50:45.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 -- start start 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 0x7f6d080d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6d0815ebe0 0x7f6d080cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d080b3d30 con 0x7f6d080b7930 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6d080d4540 con 0x7f6d0815ebe0 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.184+0000 7f6d0e971640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6d080ba7a0 con 0x7f6d080d4790 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d0cf0e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6d080b7930 0x7f6d080b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49774/0 (socket says 172.21.15.22:49774) 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d0cf0e640 1 -- 172.21.15.22:0/2314216488 learned_addr learned my addr 172.21.15.22:0/2314216488 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2314216488 >> v2:172.21.15.183:3300/0 conn(0x7f6d0815ebe0 0x7f6d080cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906161163 0 0) 0x7f6d000014a0 con 0x7f6d080b7930 2024-04-27T07:50:45.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cf0003530 con 0x7f6d080b7930 2024-04-27T07:50:45.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3186360850 0 0) 0x7f6cf0003530 con 0x7f6d080b7930 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 msgr2=0x7f6d080d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 0x7f6d080d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 >> v2:172.21.15.183:3300/0 conn(0x7f6d0815ebe0 msgr2=0x7f6d080cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 --2- 172.21.15.22:0/2314216488 >> v2:172.21.15.183:3300/0 conn(0x7f6d0815ebe0 0x7f6d080cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080d50b0 con 0x7f6d080b7930 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f6d00001bf0 con 0x7f6d080b7930 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 >> v1:172.21.15.22:6789/0 conn(0x7f6d080b7930 legacy=0x7f6d080b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 0x7f6cf00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cf0003f40 0x7f6cf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cf0004d50 0x7f6cf00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080d4540 con 0x7f6cf0003f40 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080ba7a0 con 0x7f6cf0004d50 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf000bbb0 con 0x7f6d080d4790 2024-04-27T07:50:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cf0003f40 0x7f6cf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d07fff640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 0x7f6cf00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 -- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 msgr2=0x7f6cf00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080d4790 0x7f6cf00039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 -- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cf0004d50 msgr2=0x7f6cf00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cf0004d50 0x7f6cf00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.188+0000 7f6d077fe640 1 -- 172.21.15.22:0/2314216488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf0003530 con 0x7f6cf0003f40 2024-04-27T07:50:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cf0003f40 0x7f6cf0004810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009b10 tx=0x7f6cfc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b06e716b877011 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0096b0 con 0x7f6cf0003f40 2024-04-27T07:50:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc0a6490 con 0x7f6cf0003f40 2024-04-27T07:50:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d06ffd640 1 -- 172.21.15.22:0/2314216488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a6880 con 0x7f6cf0003f40 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- 172.21.15.22:0/2314216488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cf0003f40 msgr2=0x7f6cf0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2314216488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cf0003f40 0x7f6cf0004810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009b10 tx=0x7f6cfc01cab0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0cf0e640 1 -- 172.21.15.22:0/2314216488 reap_dead start 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- 172.21.15.22:0/2314216488 shutdown_connections 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- 172.21.15.22:0/2314216488 >> 172.21.15.22:0/2314216488 conn(0x7f6d08067d10 msgr2=0x7f6d08068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- 172.21.15.22:0/2314216488 shutdown_connections 2024-04-27T07:50:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- 172.21.15.22:0/2314216488 wait complete. 2024-04-27T07:50:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 Processor -- start 2024-04-27T07:50:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- start start 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080cc920 0x7f6d08159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0815a1b0 0x7f6d0815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080ba2a0 con 0x7f6d08155250 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080ba120 con 0x7f6d0815a1b0 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.192+0000 7f6d0e971640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6d080cfda0 con 0x7f6d080cc920 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33510/0 (socket says 172.21.15.22:33510) 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 -- 172.21.15.22:0/2049522989 learned_addr learned my addr 172.21.15.22:0/2049522989 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 -- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080cc920 msgr2=0x7f6d08159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080cc920 0x7f6d08159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 -- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0815a1b0 msgr2=0x7f6d0815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0815a1b0 0x7f6d0815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 -- 172.21.15.22:0/2049522989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080cd000 con 0x7f6d08155250 2024-04-27T07:50:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d07fff640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c5c0 tx=0x7f6cf800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8015070 con 0x7f6d08155250 2024-04-27T07:50:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf8004220 con 0x7f6d08155250 2024-04-27T07:50:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8003600 con 0x7f6d08155250 2024-04-27T07:50:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d080cd2e0 con 0x7f6d08155250 2024-04-27T07:50:45.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d080cd8a0 con 0x7f6d08155250 2024-04-27T07:50:45.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0815a5d0 con 0x7f6d08155250 2024-04-27T07:50:45.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6cf80037a0 con 0x7f6d08155250 2024-04-27T07:50:45.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 0x7f6cf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6cf80863a0 con 0x7f6d08155250 2024-04-27T07:50:45.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.196+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 0x7f6cf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.200+0000 7f6d077fe640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 0x7f6cf406a840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0ad000 tx=0x7f6cfc0a9040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.200+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6cf8056cb0 con 0x7f6d08155250 2024-04-27T07:50:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.364+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d080ccd60 con 0x7f6d08155250 2024-04-27T07:50:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d04ff9640 1 -- 172.21.15.22:0/2049522989 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12860 (secure 0 0 0) 0x7f6cf80565b0 con 0x7f6d08155250 2024-04-27T07:50:45.372 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:45.372 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:44.682123+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 msgr2=0x7f6cf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 0x7f6cf406a840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0ad000 tx=0x7f6cfc0a9040 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 msgr2=0x7f6d08155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c5c0 tx=0x7f6cf800ca80 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 shutdown_connections 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6cf4068380 0x7f6cf406a840 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0815a1b0 0x7f6d0815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080cc920 0x7f6d08159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 --2- 172.21.15.22:0/2049522989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d08155250 0x7f6d08155e60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 >> 172.21.15.22:0/2049522989 conn(0x7f6d08067d10 msgr2=0x7f6d080b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 shutdown_connections 2024-04-27T07:50:45.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.368+0000 7f6d0e971640 1 -- 172.21.15.22:0/2049522989 wait complete. 2024-04-27T07:50:45.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:50:45.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:50:45.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 Processor -- start 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 -- start start 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b700d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9b70163130 0x7f9b700d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b700d5680 con 0x7f9b70161fa0 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f9b700cfe00 con 0x7f9b70163130 2024-04-27T07:50:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b754ec640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9b701635c0 con 0x7f9b700d1940 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b6e7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b700d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b6f7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9b70163130 0x7f9b700d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b6f7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9b70163130 0x7f9b700d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:59678/0 (socket says 172.21.15.22:59678) 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/1143144351 learned_addr learned my addr 172.21.15.22:0/1143144351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.520+0000 7f9b6effd640 1 --1- 172.21.15.22:0/1143144351 >> v1:172.21.15.183:6789/0 conn(0x7f9b70161fa0 0x7f9b700d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38470/0 (socket says 172.21.15.22:38470) 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 msgr2=0x7f9b700d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6f7fe640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b700d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/1143144351 >> v1:172.21.15.183:6789/0 conn(0x7f9b70161fa0 legacy=0x7f9b700d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/1143144351 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700cf4d0 con 0x7f9b70163130 2024-04-27T07:50:45.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6e7fc640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b700d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:45.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6f7fe640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b700d4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009e50 tx=0x7f9b6801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e600b2dedaf2f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6dffb640 1 -- 172.21.15.22:0/1143144351 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a5750 con 0x7f9b70163130 2024-04-27T07:50:45.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6dffb640 1 -- 172.21.15.22:0/1143144351 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b680a58f0 con 0x7f9b70163130 2024-04-27T07:50:45.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b6dffb640 1 -- 172.21.15.22:0/1143144351 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a5cc0 con 0x7f9b70163130 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 -- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 msgr2=0x7f9b700d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b700d4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009e50 tx=0x7f9b6801c9a0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 -- 172.21.15.22:0/1143144351 shutdown_connections 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b700d4f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1143144351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b700d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 -- 172.21.15.22:0/1143144351 >> 172.21.15.22:0/1143144351 conn(0x7f9b7008bfc0 msgr2=0x7f9b700d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.524+0000 7f9b754ec640 1 -- 172.21.15.22:0/1143144351 shutdown_connections 2024-04-27T07:50:45.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- 172.21.15.22:0/1143144351 wait complete. 2024-04-27T07:50:45.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 Processor -- start 2024-04-27T07:50:45.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- start start 2024-04-27T07:50:45.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b70140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b701431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b700b8000 con 0x7f9b70161fa0 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b700b7e80 con 0x7f9b70163130 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9b700b8180 con 0x7f9b700d1940 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33520/0 (socket says 172.21.15.22:33520) 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 -- 172.21.15.22:0/1974875397 learned_addr learned my addr 172.21.15.22:0/1974875397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6f7fe640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b701431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6effd640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b70140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 -- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 msgr2=0x7f9b70140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b70140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 -- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 msgr2=0x7f9b701431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b701431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 -- 172.21.15.22:0/1974875397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b70140e70 con 0x7f9b70161fa0 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6effd640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b70140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:45.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b6e7fc640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e960 tx=0x7f9b6400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400ca00 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6400cba0 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b70141150 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640059a0 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.528+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b70141710 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7004ff10 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b64005b40 con 0x7f9b70161fa0 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b577fe640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 0x7f9b5c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9b64086a10 con 0x7f9b70161fa0 2024-04-27T07:50:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b6effd640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 0x7f9b5c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.532+0000 7f9b6effd640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 0x7f9b5c06a840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000f70 tx=0x7f9b60007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.536+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b64017020 con 0x7f9b70161fa0 2024-04-27T07:50:45.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.700+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b7008c750 con 0x7f9b70161fa0 2024-04-27T07:50:45.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.700+0000 7f9b577fe640 1 -- 172.21.15.22:0/1974875397 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f9b64056c30 con 0x7f9b70161fa0 2024-04-27T07:50:45.705 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 msgr2=0x7f9b5c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 0x7f9b5c06a840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000f70 tx=0x7f9b60007d70 comp rx=0 tx=0).stop 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 msgr2=0x7f9b70155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e960 tx=0x7f9b6400ee20 comp rx=0 tx=0).stop 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 shutdown_connections 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9b5c068380 0x7f9b5c06a840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b70163130 0x7f9b701431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b70161fa0 0x7f9b70155750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 --2- 172.21.15.22:0/1974875397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9b700d1940 0x7f9b70140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 >> 172.21.15.22:0/1974875397 conn(0x7f9b7008bfc0 msgr2=0x7f9b700b98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 shutdown_connections 2024-04-27T07:50:45.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.704+0000 7f9b754ec640 1 -- 172.21.15.22:0/1974875397 wait complete. 2024-04-27T07:50:45.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-27T07:50:45.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 Processor -- start 2024-04-27T07:50:45.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 -- start start 2024-04-27T07:50:45.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00d00ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00d00d1740 0x7f00d00d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d00d3250 con 0x7f00d00d42f0 2024-04-27T07:50:45.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f00d00b88d0 con 0x7f00d00d1740 2024-04-27T07:50:45.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d7c54640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f00d00d3530 con 0x7f00d015ea90 2024-04-27T07:50:45.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d51ef640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f00d00d42f0 0x7f00d00d2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49786/0 (socket says 172.21.15.22:49786) 2024-04-27T07:50:45.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d51ef640 1 -- 172.21.15.22:0/863017140 learned_addr learned my addr 172.21.15.22:0/863017140 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00d00ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d61f1640 1 --2- 172.21.15.22:0/863017140 >> v2:172.21.15.183:3300/0 conn(0x7f00d00d1740 0x7f00d00d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337483735 0 0) 0x7f00d00d3250 con 0x7f00d00d42f0 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00bc003530 con 0x7f00d00d42f0 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3911543309 0 0) 0x7f00bc003530 con 0x7f00d00d42f0 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 msgr2=0x7f00d00ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00d00ba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 >> v2:172.21.15.183:3300/0 conn(0x7f00d00d1740 msgr2=0x7f00d00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 --2- 172.21.15.22:0/863017140 >> v2:172.21.15.183:3300/0 conn(0x7f00d00d1740 0x7f00d00d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d00d3810 con 0x7f00d00d42f0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f00c0001680 con 0x7f00d00d42f0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 >> v1:172.21.15.22:6789/0 conn(0x7f00d00d42f0 legacy=0x7f00d00d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc0039c0 0x7f00bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc004810 0x7f00bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00d00b88d0 con 0x7f00bc0039c0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00d00d3530 con 0x7f00bc004810 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc00b9d0 con 0x7f00d015ea90 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc0039c0 0x7f00bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d51ef640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc004810 0x7f00bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d61f1640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 -- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 msgr2=0x7f00bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d015ea90 0x7f00bc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 -- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc004810 msgr2=0x7f00bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:45.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc004810 0x7f00bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d59f0640 1 -- 172.21.15.22:0/863017140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc003530 con 0x7f00bc0039c0 2024-04-27T07:50:45.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.848+0000 7f00d51ef640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc004810 0x7f00bc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d59f0640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc0039c0 0x7f00bc003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009ae0 tx=0x7f00cc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaa56327d5b91d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0096b0 con 0x7f00bc0039c0 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00cc0a6460 con 0x7f00bc0039c0 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d49ee640 1 -- 172.21.15.22:0/863017140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0a6850 con 0x7f00bc0039c0 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- 172.21.15.22:0/863017140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc0039c0 msgr2=0x7f00bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 --2- 172.21.15.22:0/863017140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc0039c0 0x7f00bc003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009ae0 tx=0x7f00cc01ca70 comp rx=0 tx=0).stop 2024-04-27T07:50:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d61f1640 1 -- 172.21.15.22:0/863017140 reap_dead start 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- 172.21.15.22:0/863017140 shutdown_connections 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- 172.21.15.22:0/863017140 >> 172.21.15.22:0/863017140 conn(0x7f00d0067d40 msgr2=0x7f00d0068150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- 172.21.15.22:0/863017140 shutdown_connections 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- 172.21.15.22:0/863017140 wait complete. 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 Processor -- start 2024-04-27T07:50:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- start start 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc0039c0 0x7f00d00ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 0x7f00d00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00d00b8250 con 0x7f00d00cccb0 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00d00b80d0 con 0x7f00bc0039c0 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d7c54640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f00d00b83d0 con 0x7f00d0157a40 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d51ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d51ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33548/0 (socket says 172.21.15.22:33548) 2024-04-27T07:50:45.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.852+0000 7f00d51ef640 1 -- 172.21.15.22:0/2902344534 learned_addr learned my addr 172.21.15.22:0/2902344534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d59f0640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc0039c0 0x7f00d00ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d61f1640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 0x7f00d00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 -- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 msgr2=0x7f00d00b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 0x7f00d00b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 -- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc0039c0 msgr2=0x7f00d00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc0039c0 0x7f00d00ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 -- 172.21.15.22:0/2902344534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d00cd190 con 0x7f00d00cccb0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d61f1640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 0x7f00d00b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d51ef640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f00c000d7a0 tx=0x7f00c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0019070 con 0x7f00d00cccb0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c00074e0 con 0x7f00d00cccb0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d00cd470 con 0x7f00d00cccb0 2024-04-27T07:50:45.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0004a30 con 0x7f00d00cccb0 2024-04-27T07:50:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d00cda30 con 0x7f00d00cccb0 2024-04-27T07:50:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d0069340 con 0x7f00d00cccb0 2024-04-27T07:50:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f00c0004bd0 con 0x7f00d00cccb0 2024-04-27T07:50:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 0x7f00b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:45.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f00c0085e40 con 0x7f00d00cccb0 2024-04-27T07:50:45.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d59f0640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 0x7f00b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:45.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.856+0000 7f00d59f0640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 0x7f00b406a870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0ad000 tx=0x7f00cc0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:45.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:45.860+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00c0056750 con 0x7f00d00cccb0 2024-04-27T07:50:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.024+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f00d00b6de0 con 0x7f00d00cccb0 2024-04-27T07:50:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00c67fc640 1 -- 172.21.15.22:0/2902344534 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f00c0056050 con 0x7f00d00cccb0 2024-04-27T07:50:46.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:46.031 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-04-27T07:50:46.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:50:46.031 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-04-27T07:50:46.031 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-04-27T07:50:46.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:50:46.032 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 msgr2=0x7f00b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 0x7f00b406a870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0ad000 tx=0x7f00cc0046d0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 msgr2=0x7f00d014eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f00c000d7a0 tx=0x7f00c000dc60 comp rx=0 tx=0).stop 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 shutdown_connections 2024-04-27T07:50:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f00b40683b0 0x7f00b406a870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00d0157a40 0x7f00d00b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d00cccb0 0x7f00d014eb50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 --2- 172.21.15.22:0/2902344534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc0039c0 0x7f00d00ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 >> 172.21.15.22:0/2902344534 conn(0x7f00d0067d40 msgr2=0x7f00d015dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 shutdown_connections 2024-04-27T07:50:46.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.028+0000 7f00d7c54640 1 -- 172.21.15.22:0/2902344534 wait complete. 2024-04-27T07:50:46.050 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-27T07:50:46.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-27T07:50:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.176+0000 7fcae525b640 1 Processor -- start 2024-04-27T07:50:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.176+0000 7fcae525b640 1 -- start start 2024-04-27T07:50:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcae525b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 0x7fcae00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcae525b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcae00b05b0 0x7fcae00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcae525b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae0070130 con 0x7fcae00b0b00 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcae525b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fcae00d1150 con 0x7fcae00b05b0 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcae525b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcae006bbd0 con 0x7fcae00d0830 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcade59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcae00b0b00 0x7fcae00cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49790/0 (socket says 172.21.15.22:49790) 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcade59c640 1 -- 172.21.15.22:0/2535130982 learned_addr learned my addr 172.21.15.22:0/2535130982 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcadf59e640 1 --2- 172.21.15.22:0/2535130982 >> v2:172.21.15.183:3300/0 conn(0x7fcae00b05b0 0x7fcae00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 0x7fcae00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661294551 0 0) 0x7fcae0070130 con 0x7fcae00b0b00 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcac8003530 con 0x7fcae00b0b00 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4225233369 0 0) 0x7fcac8003530 con 0x7fcae00b0b00 2024-04-27T07:50:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 msgr2=0x7fcae00d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 0x7fcae00d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 >> v2:172.21.15.183:3300/0 conn(0x7fcae00b05b0 msgr2=0x7fcae00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 --2- 172.21.15.22:0/2535130982 >> v2:172.21.15.183:3300/0 conn(0x7fcae00b05b0 0x7fcae00ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00cd9f0 con 0x7fcae00b0b00 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fcad4001680 con 0x7fcae00b0b00 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcadf59e640 1 --2- 172.21.15.22:0/2535130982 >> v2:172.21.15.183:3300/0 conn(0x7fcae00b05b0 0x7fcae00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 >> v1:172.21.15.22:6789/0 conn(0x7fcae00b0b00 legacy=0x7fcae00cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac80039c0 0x7fcac8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac8004810 0x7fcac8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 0x7fcac80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00d1150 con 0x7fcac80039c0 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcae006bbd0 con 0x7fcac8004810 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcac800b9d0 con 0x7fcae00d0830 2024-04-27T07:50:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac80039c0 0x7fcac8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 -- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 msgr2=0x7fcac80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae00d0830 0x7fcac80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 -- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac8004810 msgr2=0x7fcac8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcade59c640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac8004810 0x7fcac8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac8004810 0x7fcac8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.180+0000 7fcaded9d640 1 -- 172.21.15.22:0/2535130982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac8003530 con 0x7fcac80039c0 2024-04-27T07:50:46.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcaded9d640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac80039c0 0x7fcac8003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009fc0 tx=0x7fcad001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100abc0521513a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2070 con 0x7fcac80039c0 2024-04-27T07:50:46.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5ae0 con 0x7fcac80039c0 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcaddd9b640 1 -- 172.21.15.22:0/2535130982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0009c10 con 0x7fcac80039c0 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- 172.21.15.22:0/2535130982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac80039c0 msgr2=0x7fcac8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 --2- 172.21.15.22:0/2535130982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac80039c0 0x7fcac8003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009fc0 tx=0x7fcad001ccc0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 -- 172.21.15.22:0/2535130982 reap_dead start 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- 172.21.15.22:0/2535130982 shutdown_connections 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- 172.21.15.22:0/2535130982 >> 172.21.15.22:0/2535130982 conn(0x7fcae009aa30 msgr2=0x7fcae00d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- 172.21.15.22:0/2535130982 shutdown_connections 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- 172.21.15.22:0/2535130982 wait complete. 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 Processor -- start 2024-04-27T07:50:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- start start 2024-04-27T07:50:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac80039c0 0x7fcae00b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae01f8430 0x7fcae01f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00cd650 con 0x7fcae01f74b0 2024-04-27T07:50:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00cbb50 con 0x7fcac80039c0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcae525b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00cd7d0 con 0x7fcae01f8430 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33572/0 (socket says 172.21.15.22:33572) 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 -- 172.21.15.22:0/4186541579 learned_addr learned my addr 172.21.15.22:0/4186541579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcaded9d640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac80039c0 0x7fcae00b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcade59c640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae01f8430 0x7fcae01f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 -- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae01f8430 msgr2=0x7fcae01f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae01f8430 0x7fcae01f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 -- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac80039c0 msgr2=0x7fcae00b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac80039c0 0x7fcae00b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.184+0000 7fcadf59e640 1 -- 172.21.15.22:0/4186541579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae01f88b0 con 0x7fcae01f74b0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcadf59e640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcae00b0a60 tx=0x7fcad800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8017070 con 0x7fcae01f74b0 2024-04-27T07:50:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad800c8c0 con 0x7fcae01f74b0 2024-04-27T07:50:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae00b5230 con 0x7fcae01f74b0 2024-04-27T07:50:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad800ccb0 con 0x7fcae01f74b0 2024-04-27T07:50:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae00b57f0 con 0x7fcae01f74b0 2024-04-27T07:50:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae004ff10 con 0x7fcae01f74b0 2024-04-27T07:50:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcad801a070 con 0x7fcae01f74b0 2024-04-27T07:50:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 0x7fcacc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fcad8086300 con 0x7fcae01f74b0 2024-04-27T07:50:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcaded9d640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 0x7fcacc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.188+0000 7fcaded9d640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 0x7fcacc06a870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcad00af000 tx=0x7fcad00a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.192+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcae01f88b0 con 0x7fcae01f74b0 2024-04-27T07:50:46.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.360+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcae01691b0 con 0x7fcae01f74b0 2024-04-27T07:50:46.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.732+0000 7fcac77fe640 1 -- 172.21.15.22:0/4186541579 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fcad8056c00 con 0x7fcae01f74b0 2024-04-27T07:50:46.737 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-04-27T07:50:46.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 msgr2=0x7fcacc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 0x7fcacc06a870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcad00af000 tx=0x7fcad00a7040 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 msgr2=0x7fcae00ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcae00b0a60 tx=0x7fcad800ec90 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 shutdown_connections 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcacc0683b0 0x7fcacc06a870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae01f74b0 0x7fcae00ae290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcae01f8430 0x7fcae01f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 --2- 172.21.15.22:0/4186541579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcac80039c0 0x7fcae00b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 >> 172.21.15.22:0/4186541579 conn(0x7fcae009aa30 msgr2=0x7fcae006ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:46.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 shutdown_connections 2024-04-27T07:50:46.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.736+0000 7fcae525b640 1 -- 172.21.15.22:0/4186541579 wait complete. 2024-04-27T07:50:46.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-27T07:50:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 Processor -- start 2024-04-27T07:50:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 -- start start 2024-04-27T07:50:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 0x7fefa81668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa80cdb10 con 0x7fefa8071f70 2024-04-27T07:50:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fefa80ce0d0 con 0x7fefa806f750 2024-04-27T07:50:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefad283640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefa80cd8c0 con 0x7fefa80cffe0 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa77fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 0x7fefa81668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:41104/0 (socket says 172.21.15.22:41104) 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 -- 172.21.15.22:0/1478940616 learned_addr learned my addr 172.21.15.22:0/1478940616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 -- 172.21.15.22:0/1478940616 >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 msgr2=0x7fefa81668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 --2- 172.21.15.22:0/1478940616 >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 0x7fefa81668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 -- 172.21.15.22:0/1478940616 >> v1:172.21.15.183:6789/0 conn(0x7fefa8071f70 legacy=0x7fefa8072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa67fc640 1 -- 172.21.15.22:0/1478940616 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa80cddf0 con 0x7fefa80cffe0 2024-04-27T07:50:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.908+0000 7fefa77fe640 1 --2- 172.21.15.22:0/1478940616 >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 0x7fefa81668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefa67fc640 1 --2- 172.21.15.22:0/1478940616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009870 tx=0x7fef9c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db09f8556b0c361 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefa5ffb640 1 -- 172.21.15.22:0/1478940616 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0b0070 con 0x7fefa80cffe0 2024-04-27T07:50:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefa5ffb640 1 -- 172.21.15.22:0/1478940616 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c0ab600 con 0x7fefa80cffe0 2024-04-27T07:50:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefa5ffb640 1 -- 172.21.15.22:0/1478940616 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0ab9d0 con 0x7fefa80cffe0 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 -- 172.21.15.22:0/1478940616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 msgr2=0x7fefa8164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 --2- 172.21.15.22:0/1478940616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009870 tx=0x7fef9c01c4d0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 -- 172.21.15.22:0/1478940616 shutdown_connections 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 --2- 172.21.15.22:0/1478940616 >> v2:172.21.15.183:3300/0 conn(0x7fefa806f750 0x7fefa81668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 --2- 172.21.15.22:0/1478940616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa80cffe0 0x7fefa8164390 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 -- 172.21.15.22:0/1478940616 >> 172.21.15.22:0/1478940616 conn(0x7fefa80b21d0 msgr2=0x7fefa80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:46.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 -- 172.21.15.22:0/1478940616 shutdown_connections 2024-04-27T07:50:46.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.912+0000 7fefad283640 1 -- 172.21.15.22:0/1478940616 wait complete. 2024-04-27T07:50:46.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 Processor -- start 2024-04-27T07:50:46.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- start start 2024-04-27T07:50:46.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa806f750 0x7fefa80ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa8071f70 0x7fefa8166110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefa80cd030 con 0x7fefa80cffe0 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefa80cceb0 con 0x7fefa8071f70 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefa8169640 con 0x7fefa806f750 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33574/0 (socket says 172.21.15.22:33574) 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 -- 172.21.15.22:0/2418858976 learned_addr learned my addr 172.21.15.22:0/2418858976 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa6ffd640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa806f750 0x7fefa80ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa67fc640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa8071f70 0x7fefa8166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 -- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa806f750 msgr2=0x7fefa80ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa806f750 0x7fefa80ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 -- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa8071f70 msgr2=0x7fefa8166110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa8071f70 0x7fefa8166110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 -- 172.21.15.22:0/2418858976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa80abf40 con 0x7fefa80cffe0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefa77fe640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefa000ef90 tx=0x7fefa0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0018020 con 0x7fefa80cffe0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0014480 con 0x7fefa80cffe0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa80ac220 con 0x7fefa80cffe0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0014870 con 0x7fefa80cffe0 2024-04-27T07:50:46.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa80ac7e0 con 0x7fefa80cffe0 2024-04-27T07:50:46.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.916+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa804ff10 con 0x7fefa80cffe0 2024-04-27T07:50:46.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.920+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fefa001b020 con 0x7fefa80cffe0 2024-04-27T07:50:46.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.920+0000 7fef8f7fe640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 0x7fef9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:46.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.920+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fefa0087500 con 0x7fefa80cffe0 2024-04-27T07:50:46.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.920+0000 7fefa6ffd640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 0x7fef9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:46.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.920+0000 7fefa6ffd640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 0x7fef9406a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b1600 tx=0x7fef98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:46.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:46.924+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fefa0022030 con 0x7fefa80cffe0 2024-04-27T07:50:47.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.088+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fefa8072cb0 con 0x7fefa80cffe0 2024-04-27T07:50:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.736+0000 7fef8f7fe640 1 -- 172.21.15.22:0/2418858976 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fefa0010040 con 0x7fefa80cffe0 2024-04-27T07:50:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-04-27T07:50:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 msgr2=0x7fef9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 0x7fef9406a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b1600 tx=0x7fef98009040 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 msgr2=0x7fefa8166650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefa000ef90 tx=0x7fefa0009f90 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 shutdown_connections 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fef94068380 0x7fef9406a840 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa80cffe0 0x7fefa8166650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa8071f70 0x7fefa8166110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 --2- 172.21.15.22:0/2418858976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa806f750 0x7fefa80ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 >> 172.21.15.22:0/2418858976 conn(0x7fefa80b21d0 msgr2=0x7fefa815ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 shutdown_connections 2024-04-27T07:50:47.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.740+0000 7fefad283640 1 -- 172.21.15.22:0/2418858976 wait complete. 2024-04-27T07:50:47.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-27T07:50:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 Processor -- start 2024-04-27T07:50:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 -- start start 2024-04-27T07:50:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8e4c09bb10 0x7f8e4c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c0d0bf0 con 0x7f8e4c06dc30 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0d1c90 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e5222d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0d1800 con 0x7f8e4c09bb10 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33582/0 (socket says 172.21.15.22:33582) 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 -- 172.21.15.22:0/574642488 learned_addr learned my addr 172.21.15.22:0/574642488 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4affd640 1 --2- 172.21.15.22:0/574642488 >> v2:172.21.15.183:3300/0 conn(0x7f8e4c09bb10 0x7f8e4c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 -- 172.21.15.22:0/574642488 >> v2:172.21.15.183:3300/0 conn(0x7f8e4c09bb10 msgr2=0x7f8e4c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 --2- 172.21.15.22:0/574642488 >> v2:172.21.15.183:3300/0 conn(0x7f8e4c09bb10 0x7f8e4c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 -- 172.21.15.22:0/574642488 >> v1:172.21.15.183:6789/0 conn(0x7f8e4c06dc30 legacy=0x7f8e4c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.912+0000 7f8e4b7fe640 1 -- 172.21.15.22:0/574642488 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c0d0ed0 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4b7fe640 1 --2- 172.21.15.22:0/574642488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06c840 tx=0x7f8e3c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bb6aadcfc448a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4a7fc640 1 -- 172.21.15.22:0/574642488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0b0070 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4a7fc640 1 -- 172.21.15.22:0/574642488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c0ab600 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4a7fc640 1 -- 172.21.15.22:0/574642488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0ab9d0 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- 172.21.15.22:0/574642488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c0ce6c0 msgr2=0x7f8e4c09b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- 172.21.15.22:0/574642488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06c840 tx=0x7f8e3c01c520 comp rx=0 tx=0).stop 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- 172.21.15.22:0/574642488 shutdown_connections 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- 172.21.15.22:0/574642488 >> v2:172.21.15.183:3300/0 conn(0x7f8e4c09bb10 0x7f8e4c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- 172.21.15.22:0/574642488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c0ce6c0 0x7f8e4c09b5d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- 172.21.15.22:0/574642488 >> 172.21.15.22:0/574642488 conn(0x7f8e4c0b4050 msgr2=0x7f8e4c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- 172.21.15.22:0/574642488 shutdown_connections 2024-04-27T07:50:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- 172.21.15.22:0/574642488 wait complete. 2024-04-27T07:50:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 Processor -- start 2024-04-27T07:50:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- start start 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c06dc30 0x7f8e4c0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e4c0ce6c0 0x7f8e4c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c06e040 con 0x7f8e4c09bb10 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d09b0 con 0x7f8e4c06dc30 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e5222d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c15ecf0 con 0x7f8e4c0ce6c0 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33596/0 (socket says 172.21.15.22:33596) 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 -- 172.21.15.22:0/3776486541 learned_addr learned my addr 172.21.15.22:0/3776486541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4b7fe640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c06dc30 0x7f8e4c0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4bfff640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e4c0ce6c0 0x7f8e4c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 -- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e4c0ce6c0 msgr2=0x7f8e4c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e4c0ce6c0 0x7f8e4c154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 -- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c06dc30 msgr2=0x7f8e4c0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c06dc30 0x7f8e4c0b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.916+0000 7f8e4affd640 1 -- 172.21.15.22:0/3776486541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c159290 con 0x7f8e4c09bb10 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e4affd640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d800 tx=0x7f8e4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000b6c0 con 0x7f8e4c09bb10 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40019070 con 0x7f8e4c09bb10 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000ba60 con 0x7f8e4c09bb10 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c0b6a70 con 0x7f8e4c09bb10 2024-04-27T07:50:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c0b7030 con 0x7f8e4c09bb10 2024-04-27T07:50:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c06c8b0 con 0x7f8e4c09bb10 2024-04-27T07:50:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8e4000bc00 con 0x7f8e4c09bb10 2024-04-27T07:50:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 0x7f8e3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f8e40086e60 con 0x7f8e4c09bb10 2024-04-27T07:50:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e4b7fe640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 0x7f8e3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.920+0000 7f8e4b7fe640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 0x7f8e3806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0ad000 tx=0x7f8e3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:47.924+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e40057470 con 0x7f8e4c09bb10 2024-04-27T07:50:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.124+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8e4c04fef0 con 0x7f8e4c09bb10 2024-04-27T07:50:48.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.792+0000 7f8e33fff640 1 -- 172.21.15.22:0/3776486541 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8e40056d70 con 0x7f8e4c09bb10 2024-04-27T07:50:48.805 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-04-27T07:50:48.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.808+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 msgr2=0x7f8e3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.808+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 0x7f8e3806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0ad000 tx=0x7f8e3c008040 comp rx=0 tx=0).stop 2024-04-27T07:50:48.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.808+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 msgr2=0x7f8e4c158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.808+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d800 tx=0x7f8e4000dcc0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 shutdown_connections 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8e38068380 0x7f8e3806a840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e4c0ce6c0 0x7f8e4c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e4c09bb10 0x7f8e4c158d50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 --2- 172.21.15.22:0/3776486541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c06dc30 0x7f8e4c0b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 >> 172.21.15.22:0/3776486541 conn(0x7f8e4c0b4050 msgr2=0x7f8e4c0c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 shutdown_connections 2024-04-27T07:50:48.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.812+0000 7f8e5222d640 1 -- 172.21.15.22:0/3776486541 wait complete. 2024-04-27T07:50:48.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:48.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.956+0000 7f0c2676c640 1 Processor -- start 2024-04-27T07:50:48.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.956+0000 7f0c2676c640 1 -- start start 2024-04-27T07:50:48.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c2676c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c2006f870 0x7f0c200cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c2676c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c200b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c2676c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c200720d0 con 0x7f0c20144dd0 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c2676c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0c20072370 con 0x7f0c2006f870 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c2676c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0c20072650 con 0x7f0c200cfc40 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c17fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c2006f870 0x7f0c200cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c1ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0c20144dd0 0x7f0c2006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49798/0 (socket says 172.21.15.22:49798) 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c1ffff640 1 -- 172.21.15.22:0/1147811049 learned_addr learned my addr 172.21.15.22:0/1147811049 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c24d09640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c200b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c17fff640 1 -- 172.21.15.22:0/1147811049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 msgr2=0x7f0c200b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c17fff640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c200b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c17fff640 1 -- 172.21.15.22:0/1147811049 >> v1:172.21.15.22:6789/0 conn(0x7f0c20144dd0 legacy=0x7f0c2006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.960+0000 7f0c17fff640 1 -- 172.21.15.22:0/1147811049 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c20072930 con 0x7f0c2006f870 2024-04-27T07:50:48.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c17fff640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c200cf700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5aeb11b6929b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:48.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1f7fe640 1 -- 172.21.15.22:0/1147811049 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080b0070 con 0x7f0c2006f870 2024-04-27T07:50:48.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1f7fe640 1 -- 172.21.15.22:0/1147811049 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c080ab600 con 0x7f0c2006f870 2024-04-27T07:50:48.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1f7fe640 1 -- 172.21.15.22:0/1147811049 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080ab9d0 con 0x7f0c2006f870 2024-04-27T07:50:48.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- 172.21.15.22:0/1147811049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 msgr2=0x7f0c200cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c200cf700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c4a0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- 172.21.15.22:0/1147811049 shutdown_connections 2024-04-27T07:50:48.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c200b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- 172.21.15.22:0/1147811049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c200cf700 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- 172.21.15.22:0/1147811049 >> 172.21.15.22:0/1147811049 conn(0x7f0c20067f10 msgr2=0x7f0c2006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- 172.21.15.22:0/1147811049 shutdown_connections 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- 172.21.15.22:0/1147811049 wait complete. 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 Processor -- start 2024-04-27T07:50:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- start start 2024-04-27T07:50:48.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c20136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 0x7f0c20136130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200b4ca0 con 0x7f0c2006f870 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200b4b20 con 0x7f0c20144dd0 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c2676c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200b4e20 con 0x7f0c200cfc40 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33624/0 (socket says 172.21.15.22:33624) 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 -- 172.21.15.22:0/3695040823 learned_addr learned my addr 172.21.15.22:0/3695040823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c17fff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c20136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c24d09640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 0x7f0c20136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 -- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 msgr2=0x7f0c20136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c20136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 -- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 msgr2=0x7f0c20136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 0x7f0c20136130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 -- 172.21.15.22:0/3695040823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2013c700 con 0x7f0c2006f870 2024-04-27T07:50:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c24d09640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 0x7f0c20136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1ffff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c3a0 tx=0x7f0c1000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10017070 con 0x7f0c2006f870 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.964+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c10005cf0 con 0x7f0c2006f870 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2013c9e0 con 0x7f0c2006f870 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c100072f0 con 0x7f0c2006f870 2024-04-27T07:50:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c20136670 con 0x7f0c2006f870 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c201f4ff0 con 0x7f0c2006f870 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c1001a070 con 0x7f0c2006f870 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c1d7fa640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 0x7f0c0006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c10085d10 con 0x7f0c2006f870 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c17fff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 0x7f0c0006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:48.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.968+0000 7f0c17fff640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 0x7f0c0006a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ad000 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:48.972+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c100562d0 con 0x7f0c2006f870 2024-04-27T07:50:49.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.136+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c20068c30 con 0x7f0c2006f870 2024-04-27T07:50:49.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.136+0000 7f0c1d7fa640 1 -- 172.21.15.22:0/3695040823 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16594 (secure 0 0 0) 0x7f0c10055bd0 con 0x7f0c2006f870 2024-04-27T07:50:49.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:49.141 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:48.129270+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 msgr2=0x7f0c0006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 0x7f0c0006a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ad000 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 msgr2=0x7f0c2013c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c3a0 tx=0x7f0c1000c860 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 shutdown_connections 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c00068380 0x7f0c0006a840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c20144dd0 0x7f0c20136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c200cfc40 0x7f0c20136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 --2- 172.21.15.22:0/3695040823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c2006f870 0x7f0c2013c000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 >> 172.21.15.22:0/3695040823 conn(0x7f0c20067f10 msgr2=0x7f0c200b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 shutdown_connections 2024-04-27T07:50:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.140+0000 7f0c2676c640 1 -- 172.21.15.22:0/3695040823 wait complete. 2024-04-27T07:50:49.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.276+0000 7fe888530640 1 Processor -- start 2024-04-27T07:50:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.276+0000 7fe888530640 1 -- start start 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe888530640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe8800ae660 0x7fe8800b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe888530640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe880164330 0x7fe880166720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe888530640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe880070710 con 0x7fe88006cbe0 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe888530640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe88006f270 con 0x7fe880164330 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe888530640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe88006e940 con 0x7fe8800ae660 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe886acd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe880164330 0x7fe880166720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe8862cc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe88006cbe0 0x7fe88006cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49802/0 (socket says 172.21.15.22:49802) 2024-04-27T07:50:49.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe8862cc640 1 -- 172.21.15.22:0/2155277954 learned_addr learned my addr 172.21.15.22:0/2155277954 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:49.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe885acb640 1 --2- 172.21.15.22:0/2155277954 >> v2:172.21.15.183:3300/0 conn(0x7fe8800ae660 0x7fe8800b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe8852ca640 1 -- 172.21.15.22:0/2155277954 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3416472659 0 0) 0x7fe880070710 con 0x7fe88006cbe0 2024-04-27T07:50:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe8852ca640 1 -- 172.21.15.22:0/2155277954 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe86c003530 con 0x7fe88006cbe0 2024-04-27T07:50:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe886acd640 1 -- 172.21.15.22:0/2155277954 >> v2:172.21.15.183:3300/0 conn(0x7fe8800ae660 msgr2=0x7fe8800b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe886acd640 1 --2- 172.21.15.22:0/2155277954 >> v2:172.21.15.183:3300/0 conn(0x7fe8800ae660 0x7fe8800b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe886acd640 1 -- 172.21.15.22:0/2155277954 >> v1:172.21.15.22:6789/0 conn(0x7fe88006cbe0 legacy=0x7fe88006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.280+0000 7fe886acd640 1 -- 172.21.15.22:0/2155277954 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88006ec20 con 0x7fe880164330 2024-04-27T07:50:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe886acd640 1 --2- 172.21.15.22:0/2155277954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe880164330 0x7fe880166720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8800742d0 tx=0x7fe87801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ee3ada6cfd4347 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe8852ca640 1 -- 172.21.15.22:0/2155277954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780b2070 con 0x7fe880164330 2024-04-27T07:50:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe8852ca640 1 -- 172.21.15.22:0/2155277954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8780a5950 con 0x7fe880164330 2024-04-27T07:50:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe8852ca640 1 -- 172.21.15.22:0/2155277954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780a5d20 con 0x7fe880164330 2024-04-27T07:50:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- 172.21.15.22:0/2155277954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe880164330 msgr2=0x7fe880166720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 --2- 172.21.15.22:0/2155277954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe880164330 0x7fe880166720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8800742d0 tx=0x7fe87801c900 comp rx=0 tx=0).stop 2024-04-27T07:50:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- 172.21.15.22:0/2155277954 shutdown_connections 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 --2- 172.21.15.22:0/2155277954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe880164330 0x7fe880166720 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 --2- 172.21.15.22:0/2155277954 >> v2:172.21.15.183:3300/0 conn(0x7fe8800ae660 0x7fe8800b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- 172.21.15.22:0/2155277954 >> 172.21.15.22:0/2155277954 conn(0x7fe880072870 msgr2=0x7fe880072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- 172.21.15.22:0/2155277954 shutdown_connections 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- 172.21.15.22:0/2155277954 wait complete. 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 Processor -- start 2024-04-27T07:50:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.284+0000 7fe888530640 1 -- start start 2024-04-27T07:50:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 0x7fe880157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe880164330 0x7fe88015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe8800b0840 con 0x7fe88006cbe0 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8800b06c0 con 0x7fe8800ae660 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe88015ca80 con 0x7fe880164330 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33652/0 (socket says 172.21.15.22:33652) 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 -- 172.21.15.22:0/1256720406 learned_addr learned my addr 172.21.15.22:0/1256720406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe885acb640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 0x7fe880157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 -- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe880164330 msgr2=0x7fe88015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe880164330 0x7fe88015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 -- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 msgr2=0x7fe880157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 0x7fe880157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 -- 172.21.15.22:0/1256720406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe880073c30 con 0x7fe88006cbe0 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe885acb640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 0x7fe880157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe8862cc640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c3c0 tx=0x7fe87c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c017070 con 0x7fe88006cbe0 2024-04-27T07:50:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c005cf0 con 0x7fe88006cbe0 2024-04-27T07:50:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88015de80 con 0x7fe88006cbe0 2024-04-27T07:50:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0072f0 con 0x7fe88006cbe0 2024-04-27T07:50:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88015e440 con 0x7fe88006cbe0 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe880069340 con 0x7fe88006cbe0 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe87c01a070 con 0x7fe88006cbe0 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 0x7fe86406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe87c085d80 con 0x7fe88006cbe0 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.288+0000 7fe885acb640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 0x7fe86406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.292+0000 7fe885acb640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 0x7fe86406a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8801f85f0 tx=0x7fe870008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.292+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe87c056330 con 0x7fe88006cbe0 2024-04-27T07:50:49.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.456+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe880166a40 con 0x7fe88006cbe0 2024-04-27T07:50:49.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe876ffd640 1 -- 172.21.15.22:0/1256720406 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16594 (secure 0 0 0) 0x7fe87c055c30 con 0x7fe88006cbe0 2024-04-27T07:50:49.463 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:49.463 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:48.129270+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 msgr2=0x7fe86406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 0x7fe86406a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8801f85f0 tx=0x7fe870008040 comp rx=0 tx=0).stop 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 msgr2=0x7fe880152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c3c0 tx=0x7fe87c00c880 comp rx=0 tx=0).stop 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 shutdown_connections 2024-04-27T07:50:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe864068380 0x7fe86406a840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe880164330 0x7fe88015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8800ae660 0x7fe880157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 --2- 172.21.15.22:0/1256720406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe88006cbe0 0x7fe880152df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 >> 172.21.15.22:0/1256720406 conn(0x7fe880072870 msgr2=0x7fe8800b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 shutdown_connections 2024-04-27T07:50:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.460+0000 7fe888530640 1 -- 172.21.15.22:0/1256720406 wait complete. 2024-04-27T07:50:49.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-27T07:50:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 Processor -- start 2024-04-27T07:50:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 -- start start 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff038161fa0 0x7ff0380d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 0x7ff0380d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0381635a0 con 0x7ff038163130 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff0380cfe00 con 0x7ff038161fa0 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff0402ed640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff0380d5680 con 0x7ff0380d1940 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e089640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff038161fa0 0x7ff0380d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e089640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff038161fa0 0x7ff0380d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33664/0 (socket says 172.21.15.22:33664) 2024-04-27T07:50:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e089640 1 -- 172.21.15.22:0/3171641705 learned_addr learned my addr 172.21.15.22:0/3171641705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e089640 1 -- 172.21.15.22:0/3171641705 >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 msgr2=0x7ff0380d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e88a640 1 --2- 172.21.15.22:0/3171641705 >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 0x7ff0380d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.612+0000 7ff03e089640 1 --2- 172.21.15.22:0/3171641705 >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 0x7ff0380d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03e089640 1 -- 172.21.15.22:0/3171641705 >> v1:172.21.15.183:6789/0 conn(0x7ff038163130 legacy=0x7ff0380d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03e089640 1 -- 172.21.15.22:0/3171641705 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0380cf4d0 con 0x7ff038161fa0 2024-04-27T07:50:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03e88a640 1 --2- 172.21.15.22:0/3171641705 >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 0x7ff0380d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03e089640 1 --2- 172.21.15.22:0/3171641705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038161fa0 0x7ff0380d0290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff03808dc40 tx=0x7ff03401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c19d645181f483 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03d087640 1 -- 172.21.15.22:0/3171641705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340b0070 con 0x7ff038161fa0 2024-04-27T07:50:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03d087640 1 -- 172.21.15.22:0/3171641705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0340ab600 con 0x7ff038161fa0 2024-04-27T07:50:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff03d087640 1 -- 172.21.15.22:0/3171641705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340ab9d0 con 0x7ff038161fa0 2024-04-27T07:50:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 -- 172.21.15.22:0/3171641705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038161fa0 msgr2=0x7ff0380d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 --2- 172.21.15.22:0/3171641705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038161fa0 0x7ff0380d0290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff03808dc40 tx=0x7ff03401c270 comp rx=0 tx=0).stop 2024-04-27T07:50:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 -- 172.21.15.22:0/3171641705 shutdown_connections 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 --2- 172.21.15.22:0/3171641705 >> v2:172.21.15.183:3300/0 conn(0x7ff0380d1940 0x7ff0380d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 --2- 172.21.15.22:0/3171641705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038161fa0 0x7ff0380d0290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 -- 172.21.15.22:0/3171641705 >> 172.21.15.22:0/3171641705 conn(0x7ff03808bfc0 msgr2=0x7ff0380d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 -- 172.21.15.22:0/3171641705 shutdown_connections 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.616+0000 7ff0402ed640 1 -- 172.21.15.22:0/3171641705 wait complete. 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 Processor -- start 2024-04-27T07:50:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- start start 2024-04-27T07:50:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0380d1940 0x7ff03815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff038161fa0 0x7ff038154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0380b8000 con 0x7ff038163130 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0380b7e80 con 0x7ff0380d1940 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff0380b8180 con 0x7ff038161fa0 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33668/0 (socket says 172.21.15.22:33668) 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 -- 172.21.15.22:0/1805744547 learned_addr learned my addr 172.21.15.22:0/1805744547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e089640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0380d1940 0x7ff03815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03d888640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff038161fa0 0x7ff038154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 -- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff038161fa0 msgr2=0x7ff038154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff038161fa0 0x7ff038154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 -- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0380d1940 msgr2=0x7ff03815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0380d1940 0x7ff03815e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 -- 172.21.15.22:0/1805744547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0380c9630 con 0x7ff038163130 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff03e88a640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff03000cdb0 tx=0x7ff03000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030003ba0 con 0x7ff038163130 2024-04-27T07:50:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff030003d60 con 0x7ff038163130 2024-04-27T07:50:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03815ee30 con 0x7ff038163130 2024-04-27T07:50:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030004260 con 0x7ff038163130 2024-04-27T07:50:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0380ce250 con 0x7ff038163130 2024-04-27T07:50:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03804ff10 con 0x7ff038163130 2024-04-27T07:50:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff030004400 con 0x7ff038163130 2024-04-27T07:50:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.620+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff030087490 con 0x7ff038163130 2024-04-27T07:50:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.624+0000 7ff03e089640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.624+0000 7ff03e089640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 0x7ff01c06a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ad000 tx=0x7ff034008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:49.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.628+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff030014030 con 0x7ff038163130 2024-04-27T07:50:49.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:49.832+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff0380d1010 con 0x7ff038163130 2024-04-27T07:50:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.792+0000 7ff02effd640 1 -- 172.21.15.22:0/1805744547 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7ff0300579b0 con 0x7ff038163130 2024-04-27T07:50:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 msgr2=0x7ff01c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 0x7ff01c06a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ad000 tx=0x7ff034008040 comp rx=0 tx=0).stop 2024-04-27T07:50:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 msgr2=0x7ff0381554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff03000cdb0 tx=0x7ff03000a650 comp rx=0 tx=0).stop 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 shutdown_connections 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff038163130 0x7ff0381554b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff038161fa0 0x7ff038154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 --2- 172.21.15.22:0/1805744547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0380d1940 0x7ff03815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 >> 172.21.15.22:0/1805744547 conn(0x7ff03808bfc0 msgr2=0x7ff0380b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 shutdown_connections 2024-04-27T07:50:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.796+0000 7ff0402ed640 1 -- 172.21.15.22:0/1805744547 wait complete. 2024-04-27T07:50:50.820 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-27T07:50:50.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-27T07:50:50.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 Processor -- start 2024-04-27T07:50:50.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 -- start start 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa4840b2a70 0x7fa48415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840cbfb0 con 0x7fa4840cd050 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840cc290 con 0x7fa4840cd470 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa489912640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa4840cc570 con 0x7fa4840b2a70 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa4840b2a70 0x7fa48415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:50.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa4840b2a70 0x7fa48415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44784/0 (socket says 172.21.15.22:44784) 2024-04-27T07:50:50.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 -- 172.21.15.22:0/1481069698 learned_addr learned my addr 172.21.15.22:0/1481069698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:50.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 -- 172.21.15.22:0/1481069698 >> v1:172.21.15.183:6789/0 conn(0x7fa4840cd470 legacy=0x7fa4840b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:50.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 -- 172.21.15.22:0/1481069698 >> v1:172.21.15.22:6789/0 conn(0x7fa4840cd050 legacy=0x7fa48415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.948+0000 7fa482ffd640 1 -- 172.21.15.22:0/1481069698 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840cc850 con 0x7fa4840b2a70 2024-04-27T07:50:50.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa482ffd640 1 --2- 172.21.15.22:0/1481069698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840b2a70 0x7fa48415f460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e892dc4c635e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:50.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa4827fc640 1 -- 172.21.15.22:0/1481069698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740b0070 con 0x7fa4840b2a70 2024-04-27T07:50:50.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa4827fc640 1 -- 172.21.15.22:0/1481069698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4740ab400 con 0x7fa4840b2a70 2024-04-27T07:50:50.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa4827fc640 1 -- 172.21.15.22:0/1481069698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740ab7d0 con 0x7fa4840b2a70 2024-04-27T07:50:50.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- 172.21.15.22:0/1481069698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840b2a70 msgr2=0x7fa48415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 --2- 172.21.15.22:0/1481069698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840b2a70 0x7fa48415f460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47401c490 comp rx=0 tx=0).stop 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- 172.21.15.22:0/1481069698 shutdown_connections 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 --2- 172.21.15.22:0/1481069698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840b2a70 0x7fa48415f460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- 172.21.15.22:0/1481069698 >> 172.21.15.22:0/1481069698 conn(0x7fa48409aa30 msgr2=0x7fa4840b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- 172.21.15.22:0/1481069698 shutdown_connections 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- 172.21.15.22:0/1481069698 wait complete. 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 Processor -- start 2024-04-27T07:50:50.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- start start 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4840b2a70 0x7fa48415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 0x7fa48415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa484169930 con 0x7fa4840cd050 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841697b0 con 0x7fa4840b2a70 2024-04-27T07:50:50.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa489912640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa484169ab0 con 0x7fa4840cd470 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44800/0 (socket says 172.21.15.22:44800) 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 -- 172.21.15.22:0/2182940168 learned_addr learned my addr 172.21.15.22:0/2182940168 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa482ffd640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4840b2a70 0x7fa48415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa4837fe640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 0x7fa48415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 -- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 msgr2=0x7fa48415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 0x7fa48415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 -- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4840b2a70 msgr2=0x7fa48415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4840b2a70 0x7fa48415b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa47bfff640 1 -- 172.21.15.22:0/2182940168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa484146330 con 0x7fa4840cd050 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.952+0000 7fa4837fe640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 0x7fa48415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa47bfff640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c420 tx=0x7fa46c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0052d0 con 0x7fa4840cd050 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c005470 con 0x7fa4840cd050 2024-04-27T07:50:50.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa484146610 con 0x7fa4840cd050 2024-04-27T07:50:50.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007260 con 0x7fa4840cd050 2024-04-27T07:50:50.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa484146bd0 con 0x7fa4840cd050 2024-04-27T07:50:50.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48404ff10 con 0x7fa4840cd050 2024-04-27T07:50:50.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa46c007400 con 0x7fa4840cd050 2024-04-27T07:50:50.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 0x7fa46406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:50.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa46c085ec0 con 0x7fa4840cd050 2024-04-27T07:50:50.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa482ffd640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 0x7fa46406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:50.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.956+0000 7fa482ffd640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 0x7fa46406a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4740ad000 tx=0x7fa4740a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:50.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:50.960+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa46c056480 con 0x7fa4840cd050 2024-04-27T07:50:51.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.156+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa4840b1050 con 0x7fa4840cd050 2024-04-27T07:50:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.808+0000 7fa480ff9640 1 -- 172.21.15.22:0/2182940168 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fa46c055d80 con 0x7fa4840cd050 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 msgr2=0x7fa46406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 0x7fa46406a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4740ad000 tx=0x7fa4740a6000 comp rx=0 tx=0).stop 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 msgr2=0x7fa4840c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c420 tx=0x7fa46c00c8e0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 shutdown_connections 2024-04-27T07:50:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa464068380 0x7fa46406a840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4840cd470 0x7fa48415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4840cd050 0x7fa4840c6320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 --2- 172.21.15.22:0/2182940168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4840b2a70 0x7fa48415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 >> 172.21.15.22:0/2182940168 conn(0x7fa48409aa30 msgr2=0x7fa4840d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 shutdown_connections 2024-04-27T07:50:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.812+0000 7fa489912640 1 -- 172.21.15.22:0/2182940168 wait complete. 2024-04-27T07:50:51.838 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-27T07:50:51.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:50:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 Processor -- start 2024-04-27T07:50:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 -- start start 2024-04-27T07:50:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f08800afd20 0x7f08801667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08800b3f00 con 0x7f088015ecf0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08801602c0 con 0x7f0880070f30 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f0885815640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0880070490 con 0x7f08800afd20 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f088015ecf0 0x7f088015f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36056/0 (socket says 172.21.15.22:36056) 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087effd640 1 -- 172.21.15.22:0/2287231003 learned_addr learned my addr 172.21.15.22:0/2287231003 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2287231003 >> v2:172.21.15.183:3300/0 conn(0x7f08800afd20 0x7f08801667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713646304 0 0) 0x7f0870001120 con 0x7f088015ecf0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0868005740 con 0x7f088015ecf0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 482003222 0 0) 0x7f0868005740 con 0x7f088015ecf0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 >> v2:172.21.15.183:3300/0 conn(0x7f08800afd20 msgr2=0x7f08801667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 --2- 172.21.15.22:0/2287231003 >> v2:172.21.15.183:3300/0 conn(0x7f08800afd20 0x7f08801667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 >> v1:172.21.15.183:6789/0 conn(0x7f0880070f30 legacy=0x7f088015fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800706e0 con 0x7f088015ecf0 2024-04-27T07:50:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0870001850 con 0x7f088015ecf0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 >> v1:172.21.15.22:6789/0 conn(0x7f088015ecf0 legacy=0x7f088015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0868005bd0 0x7f0868005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0868006a20 0x7f0868006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f086800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0880070490 con 0x7f08680073e0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f086800e280 con 0x7f0868006a20 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f086800e480 con 0x7f0868005bd0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f086800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087effd640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0868005bd0 0x7f0868005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087e7fc640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0868006a20 0x7f0868006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 -- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0868005bd0 msgr2=0x7f0868005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0868005bd0 0x7f0868005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 -- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0868006a20 msgr2=0x7f0868006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0868006a20 0x7f0868006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087f7fe640 1 -- 172.21.15.22:0/2287231003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0868005740 con 0x7f08680073e0 2024-04-27T07:50:51.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.964+0000 7f087e7fc640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0868006a20 0x7f0868006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f087f7fe640 1 -- 172.21.15.22:0/2287231003 reap_dead start 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f086800dbe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0878009f90 tx=0x7f087801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b7e08f8710b403 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878009c10 con 0x7f08680073e0 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08780a5c90 con 0x7f08680073e0 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f087dffb640 1 -- 172.21.15.22:0/2287231003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878003be0 con 0x7f08680073e0 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 msgr2=0x7f086800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f086800dbe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0878009f90 tx=0x7f087801cc80 comp rx=0 tx=0).stop 2024-04-27T07:50:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- 172.21.15.22:0/2287231003 shutdown_connections 2024-04-27T07:50:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 --2- 172.21.15.22:0/2287231003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f086800dbe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- 172.21.15.22:0/2287231003 >> 172.21.15.22:0/2287231003 conn(0x7f08800b3020 msgr2=0x7f08800b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- 172.21.15.22:0/2287231003 shutdown_connections 2024-04-27T07:50:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- 172.21.15.22:0/2287231003 wait complete. 2024-04-27T07:50:51.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 Processor -- start 2024-04-27T07:50:51.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 -- start start 2024-04-27T07:50:51.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.968+0000 7f0885815640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08801f55b0 0x7f0880157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08800cbdf0 0x7f08801f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08800cfc50 con 0x7f08680073e0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08800cffc0 con 0x7f08801f55b0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f088015f0d0 con 0x7f08800cbdf0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44810/0 (socket says 172.21.15.22:44810) 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 -- 172.21.15.22:0/2704890961 learned_addr learned my addr 172.21.15.22:0/2704890961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087e7fc640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08801f55b0 0x7f0880157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087f7fe640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08800cbdf0 0x7f08801f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 -- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08800cbdf0 msgr2=0x7f08801f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08800cbdf0 0x7f08801f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 -- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08801f55b0 msgr2=0x7f0880157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08801f55b0 0x7f0880157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 -- 172.21.15.22:0/2704890961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801f5d90 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087effd640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f087000d730 tx=0x7f087000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870019070 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08700074e0 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801f6070 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004a30 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801f6630 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0880069340 con 0x7f08680073e0 2024-04-27T07:50:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f087001c070 con 0x7f08680073e0 2024-04-27T07:50:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 0x7f086c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0870086360 con 0x7f08680073e0 2024-04-27T07:50:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.972+0000 7f087e7fc640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 0x7f086c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.976+0000 7f087e7fc640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 0x7f086c06a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f088015e2c0 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:51.976+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08800b3f00 con 0x7f08680073e0 2024-04-27T07:50:52.123 INFO:tasks.ceph.mds.b.smithi183.stderr:2024-04-27T07:50:52.121+0000 7f1ec98f5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:50:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.180+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0880073cd0 con 0x7f08680073e0 2024-04-27T07:50:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0867fff640 1 -- 172.21.15.22:0/2704890961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f08700562c0 con 0x7f08680073e0 2024-04-27T07:50:52.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:52.188 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:50:52.188 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 msgr2=0x7f086c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 0x7f086c06a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f088015e2c0 tx=0x7f0874009040 comp rx=0 tx=0).stop 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 msgr2=0x7f08800c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f087000d730 tx=0x7f087000dbf0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 shutdown_connections 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f086c068380 0x7f086c06a840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08800cbdf0 0x7f08801f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08801f55b0 0x7f0880157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 --2- 172.21.15.22:0/2704890961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08680073e0 0x7f08800c5a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 >> 172.21.15.22:0/2704890961 conn(0x7f08800b3020 msgr2=0x7f088006f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:52.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 shutdown_connections 2024-04-27T07:50:52.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.184+0000 7f0885815640 1 -- 172.21.15.22:0/2704890961 wait complete. 2024-04-27T07:50:52.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 Processor -- start 2024-04-27T07:50:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 -- start start 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f780d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0f7815ebe0 0x7f0f780cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 0x7f0f780b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0f780d4540 con 0x7f0f7815ebe0 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0f780b3d30 con 0x7f0f780b7930 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f7d2e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0f780ba7a0 con 0x7f0f780d4790 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0f7815ebe0 0x7f0f780cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0f7815ebe0 0x7f0f780cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44818/0 (socket says 172.21.15.22:44818) 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 -- 172.21.15.22:0/3660077299 learned_addr learned my addr 172.21.15.22:0/3660077299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f777fe640 1 --2- 172.21.15.22:0/3660077299 >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 0x7f0f780b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f76ffd640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f780d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 -- 172.21.15.22:0/3660077299 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 msgr2=0x7f0f780d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f780d4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 -- 172.21.15.22:0/3660077299 >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 msgr2=0x7f0f780b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 --2- 172.21.15.22:0/3660077299 >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 0x7f0f780b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f767fc640 1 -- 172.21.15.22:0/3660077299 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f780d50b0 con 0x7f0f7815ebe0 2024-04-27T07:50:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.320+0000 7f0f777fe640 1 --2- 172.21.15.22:0/3660077299 >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 0x7f0f780b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f767fc640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f7815ebe0 0x7f0f780cf860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f780d1250 tx=0x7f0f6c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c4e29398ff32d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f75ffb640 1 -- 172.21.15.22:0/3660077299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0b0070 con 0x7f0f7815ebe0 2024-04-27T07:50:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f75ffb640 1 -- 172.21.15.22:0/3660077299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c0ab600 con 0x7f0f7815ebe0 2024-04-27T07:50:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f75ffb640 1 -- 172.21.15.22:0/3660077299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0ab9d0 con 0x7f0f7815ebe0 2024-04-27T07:50:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/3660077299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f7815ebe0 msgr2=0x7f0f780cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f7815ebe0 0x7f0f780cf860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f780d1250 tx=0x7f0f6c01c1a0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/3660077299 shutdown_connections 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/3660077299 >> v2:172.21.15.183:3300/0 conn(0x7f0f780b7930 0x7f0f780b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f7815ebe0 0x7f0f780cf860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/3660077299 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f780d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/3660077299 >> 172.21.15.22:0/3660077299 conn(0x7f0f78067d10 msgr2=0x7f0f78068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/3660077299 shutdown_connections 2024-04-27T07:50:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/3660077299 wait complete. 2024-04-27T07:50:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 Processor -- start 2024-04-27T07:50:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- start start 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f780b7930 0x7f0f7814d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f7815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f780ba2a0 con 0x7f0f781557d0 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f780ba120 con 0x7f0f780b7930 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f7d2e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0f780cfda0 con 0x7f0f780d4790 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44830/0 (socket says 172.21.15.22:44830) 2024-04-27T07:50:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 -- 172.21.15.22:0/1508302535 learned_addr learned my addr 172.21.15.22:0/1508302535 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f76ffd640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f780b7930 0x7f0f7814d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f767fc640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f7815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 -- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 msgr2=0x7f0f7815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f7815dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 -- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f780b7930 msgr2=0x7f0f7814d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f780b7930 0x7f0f7814d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 -- 172.21.15.22:0/1508302535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7815b440 con 0x7f0f781557d0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f767fc640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f7815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f777fe640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cd10 tx=0x7f0f7000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.324+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000f680 con 0x7f0f781557d0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7000f840 con 0x7f0f781557d0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7815b720 con 0x7f0f781557d0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000fc30 con 0x7f0f781557d0 2024-04-27T07:50:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f780b5270 con 0x7f0f781557d0 2024-04-27T07:50:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f78068c30 con 0x7f0f781557d0 2024-04-27T07:50:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0f700213f0 con 0x7f0f781557d0 2024-04-27T07:50:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f5f7fe640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 0x7f0f6406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f70087600 con 0x7f0f781557d0 2024-04-27T07:50:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f76ffd640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 0x7f0f6406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.328+0000 7f0f76ffd640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 0x7f0f6406a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a8d0 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.332+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f70014030 con 0x7f0f781557d0 2024-04-27T07:50:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.488+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f78068390 con 0x7f0f781557d0 2024-04-27T07:50:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.492+0000 7f0f5f7fe640 1 -- 172.21.15.22:0/1508302535 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f0f70057bc0 con 0x7f0f781557d0 2024-04-27T07:50:52.497 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:52.497 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 msgr2=0x7f0f6406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 0x7f0f6406a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a8d0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 msgr2=0x7f0f780b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cd10 tx=0x7f0f7000a650 comp rx=0 tx=0).stop 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 shutdown_connections 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0f640683b0 0x7f0f6406a870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f781557d0 0x7f0f780b5a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f780d4790 0x7f0f7815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 --2- 172.21.15.22:0/1508302535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f780b7930 0x7f0f7814d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 >> 172.21.15.22:0/1508302535 conn(0x7f0f78067d10 msgr2=0x7f0f7815c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 shutdown_connections 2024-04-27T07:50:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.496+0000 7f0f7d2e9640 1 -- 172.21.15.22:0/1508302535 wait complete. 2024-04-27T07:50:52.515 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-27T07:50:52.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:50:52.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 Processor -- start 2024-04-27T07:50:52.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 -- start start 2024-04-27T07:50:52.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 0x7f29a8160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a815ce60 con 0x7f29a80b72c0 2024-04-27T07:50:52.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f29a8161350 con 0x7f29a815fd50 2024-04-27T07:50:52.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29af0fc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f29a815d120 con 0x7f29a806c2e0 2024-04-27T07:50:52.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ad699640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ace98640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 0x7f29a8160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ace98640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 0x7f29a8160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54402/0 (socket says 172.21.15.22:54402) 2024-04-27T07:50:52.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ace98640 1 -- 172.21.15.22:0/3522785936 learned_addr learned my addr 172.21.15.22:0/3522785936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:52.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ad699640 1 -- 172.21.15.22:0/3522785936 >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 msgr2=0x7f29a8160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:52.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ad699640 1 --2- 172.21.15.22:0/3522785936 >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 0x7f29a8160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ad699640 1 -- 172.21.15.22:0/3522785936 >> v1:172.21.15.183:6789/0 conn(0x7f29a80b72c0 legacy=0x7f29a806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.636+0000 7f29ad699640 1 -- 172.21.15.22:0/3522785936 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a815d400 con 0x7f29a806c2e0 2024-04-27T07:50:52.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29ad699640 1 --2- 172.21.15.22:0/3522785936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a815e1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097c0 tx=0x7f29a001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b3c73df8ecfdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29a77fe640 1 -- 172.21.15.22:0/3522785936 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00b0070 con 0x7f29a806c2e0 2024-04-27T07:50:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29a77fe640 1 -- 172.21.15.22:0/3522785936 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a00ab600 con 0x7f29a806c2e0 2024-04-27T07:50:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29a77fe640 1 -- 172.21.15.22:0/3522785936 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00ab9d0 con 0x7f29a806c2e0 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- 172.21.15.22:0/3522785936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 msgr2=0x7f29a815e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 --2- 172.21.15.22:0/3522785936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a815e1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097c0 tx=0x7f29a001c4a0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- 172.21.15.22:0/3522785936 shutdown_connections 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 --2- 172.21.15.22:0/3522785936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a815e1d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 --2- 172.21.15.22:0/3522785936 >> v2:172.21.15.183:3300/0 conn(0x7f29a815fd50 0x7f29a8160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- 172.21.15.22:0/3522785936 >> 172.21.15.22:0/3522785936 conn(0x7f29a809aa30 msgr2=0x7f29a80b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- 172.21.15.22:0/3522785936 shutdown_connections 2024-04-27T07:50:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- 172.21.15.22:0/3522785936 wait complete. 2024-04-27T07:50:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 Processor -- start 2024-04-27T07:50:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.640+0000 7f29af0fc640 1 -- start start 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a8155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a80b72c0 0x7f29a814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f29a815f0a0 con 0x7f29a815fd50 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29a815ef20 con 0x7f29a80b72c0 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f29a815f220 con 0x7f29a806c2e0 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44834/0 (socket says 172.21.15.22:44834) 2024-04-27T07:50:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 -- 172.21.15.22:0/4068328589 learned_addr learned my addr 172.21.15.22:0/4068328589 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a7fff640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a80b72c0 0x7f29a814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ace98640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a8155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 -- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 msgr2=0x7f29a8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a8155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 -- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a80b72c0 msgr2=0x7f29a814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a80b72c0 0x7f29a814ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 -- 172.21.15.22:0/4068328589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80d07f0 con 0x7f29a815fd50 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ad699640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29a001a0d0 tx=0x7f29a0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00abe60 con 0x7f29a815fd50 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a00b9070 con 0x7f29a815fd50 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a8150f60 con 0x7f29a815fd50 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00b0040 con 0x7f29a815fd50 2024-04-27T07:50:52.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a80d0510 con 0x7f29a815fd50 2024-04-27T07:50:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8069340 con 0x7f29a815fd50 2024-04-27T07:50:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29a00b4650 con 0x7f29a815fd50 2024-04-27T07:50:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 0x7f299406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f29a0120560 con 0x7f29a815fd50 2024-04-27T07:50:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.644+0000 7f29ace98640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 0x7f299406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:52.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.648+0000 7f29ace98640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 0x7f299406a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2998009ce0 tx=0x7f299800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.648+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29a0016090 con 0x7f29a815fd50 2024-04-27T07:50:52.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.808+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29a80d12a0 con 0x7f2994068380 2024-04-27T07:50:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.832+0000 7f29a57fa640 1 -- 172.21.15.22:0/4068328589 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7f29a80d12a0 con 0x7f2994068380 2024-04-27T07:50:52.838 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:50:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 msgr2=0x7f299406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 0x7f299406a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2998009ce0 tx=0x7f299800b040 comp rx=0 tx=0).stop 2024-04-27T07:50:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 msgr2=0x7f29a814b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29a001a0d0 tx=0x7f29a0019240 comp rx=0 tx=0).stop 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 shutdown_connections 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2994068380 0x7f299406a840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29a815fd50 0x7f29a814b390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a80b72c0 0x7f29a814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 --2- 172.21.15.22:0/4068328589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29a806c2e0 0x7f29a8155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 >> 172.21.15.22:0/4068328589 conn(0x7f29a809aa30 msgr2=0x7f29a8070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 shutdown_connections 2024-04-27T07:50:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:52.836+0000 7f29af0fc640 1 -- 172.21.15.22:0/4068328589 wait complete. 2024-04-27T07:50:52.856 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T07:50:52.148796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235484,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739236,"statfs":{"total":773094113280,"available":772852977664,"internally_reserved":0,"allocated":2535424,"data_stored":1590935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.399868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745982+0000","last_change":"2024-04-27T07:50:46.745982+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745982+0000","last_undegraded":"2024-04-27T07:50:46.745982+0000","last_fullsized":"2024-04-27T07:50:46.745982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750046+0000","last_change":"2024-04-27T07:50:46.750046+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750046+0000","last_undegraded":"2024-04-27T07:50:46.750046+0000","last_fullsized":"2024-04-27T07:50:46.750046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745425+0000","last_change":"2024-04-27T07:50:46.745425+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745425+0000","last_undegraded":"2024-04-27T07:50:46.745425+0000","last_fullsized":"2024-04-27T07:50:46.745425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750203+0000","last_change":"2024-04-27T07:50:46.750203+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750203+0000","last_undegraded":"2024-04-27T07:50:46.750203+0000","last_fullsized":"2024-04-27T07:50:46.750203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754693+0000","last_change":"2024-04-27T07:50:46.754693+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754693+0000","last_undegraded":"2024-04-27T07:50:46.754693+0000","last_fullsized":"2024-04-27T07:50:46.754693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751947+0000","last_change":"2024-04-27T07:50:46.751947+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751947+0000","last_undegraded":"2024-04-27T07:50:46.751947+0000","last_fullsized":"2024-04-27T07:50:46.751947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743673+0000","last_change":"2024-04-27T07:50:46.743673+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743673+0000","last_undegraded":"2024-04-27T07:50:46.743673+0000","last_fullsized":"2024-04-27T07:50:46.743673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751777+0000","last_change":"2024-04-27T07:50:46.751777+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751777+0000","last_undegraded":"2024-04-27T07:50:46.751777+0000","last_fullsized":"2024-04-27T07:50:46.751777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745246+0000","last_change":"2024-04-27T07:50:46.745246+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745246+0000","last_undegraded":"2024-04-27T07:50:46.745246+0000","last_fullsized":"2024-04-27T07:50:46.745246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751216+0000","last_change":"2024-04-27T07:50:46.751216+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751216+0000","last_undegraded":"2024-04-27T07:50:46.751216+0000","last_fullsized":"2024-04-27T07:50:46.751216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750082+0000","last_change":"2024-04-27T07:50:46.750082+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750082+0000","last_undegraded":"2024-04-27T07:50:46.750082+0000","last_fullsized":"2024-04-27T07:50:46.750082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753618+0000","last_change":"2024-04-27T07:50:46.753618+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753618+0000","last_undegraded":"2024-04-27T07:50:46.753618+0000","last_fullsized":"2024-04-27T07:50:46.753618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.748355+0000","last_change":"2024-04-27T07:50:46.748355+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.748355+0000","last_undegraded":"2024-04-27T07:50:46.748355+0000","last_fullsized":"2024-04-27T07:50:46.748355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754546+0000","last_change":"2024-04-27T07:50:46.754546+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754546+0000","last_undegraded":"2024-04-27T07:50:46.754546+0000","last_fullsized":"2024-04-27T07:50:46.754546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741421+0000","last_change":"2024-04-27T07:50:46.741421+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741421+0000","last_undegraded":"2024-04-27T07:50:46.741421+0000","last_fullsized":"2024-04-27T07:50:46.741421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753628+0000","last_change":"2024-04-27T07:50:46.753628+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753628+0000","last_undegraded":"2024-04-27T07:50:46.753628+0000","last_fullsized":"2024-04-27T07:50:46.753628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-27T07:50:46.752316+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:46.752316+0000","last_peered":"2024-04-27T07:50:46.752316+0000","last_clean":"2024-04-27T07:50:46.752316+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:46.752316+0000","last_undegraded":"2024-04-27T07:50:46.752316+0000","last_fullsized":"2024-04-27T07:50:46.752316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752306+0000","last_change":"2024-04-27T07:50:46.752306+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752306+0000","last_undegraded":"2024-04-27T07:50:46.752306+0000","last_fullsized":"2024-04-27T07:50:46.752306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743486+0000","last_change":"2024-04-27T07:50:46.743486+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743486+0000","last_undegraded":"2024-04-27T07:50:46.743486+0000","last_fullsized":"2024-04-27T07:50:46.743486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753620+0000","last_change":"2024-04-27T07:50:46.753620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753620+0000","last_undegraded":"2024-04-27T07:50:46.753620+0000","last_fullsized":"2024-04-27T07:50:46.753620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.749733+0000","last_change":"2024-04-27T07:50:46.749733+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.749733+0000","last_undegraded":"2024-04-27T07:50:46.749733+0000","last_fullsized":"2024-04-27T07:50:46.749733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741620+0000","last_change":"2024-04-27T07:50:46.741620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741620+0000","last_undegraded":"2024-04-27T07:50:46.741620+0000","last_fullsized":"2024-04-27T07:50:46.741620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743644+0000","last_change":"2024-04-27T07:50:46.743644+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743644+0000","last_undegraded":"2024-04-27T07:50:46.743644+0000","last_fullsized":"2024-04-27T07:50:46.743644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741748+0000","last_change":"2024-04-27T07:50:46.741748+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741748+0000","last_undegraded":"2024-04-27T07:50:46.741748+0000","last_fullsized":"2024-04-27T07:50:46.741748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752940+0000","last_change":"2024-04-27T07:50:46.752940+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752940+0000","last_undegraded":"2024-04-27T07:50:46.752940+0000","last_fullsized":"2024-04-27T07:50:46.752940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745623+0000","last_change":"2024-04-27T07:50:46.745623+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745623+0000","last_undegraded":"2024-04-27T07:50:46.745623+0000","last_fullsized":"2024-04-27T07:50:46.745623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741257+0000","last_change":"2024-04-27T07:50:46.741257+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741257+0000","last_undegraded":"2024-04-27T07:50:46.741257+0000","last_fullsized":"2024-04-27T07:50:46.741257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741427+0000","last_change":"2024-04-27T07:50:46.741427+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741427+0000","last_undegraded":"2024-04-27T07:50:46.741427+0000","last_fullsized":"2024-04-27T07:50:46.741427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743511+0000","last_change":"2024-04-27T07:50:46.743511+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743511+0000","last_undegraded":"2024-04-27T07:50:46.743511+0000","last_fullsized":"2024-04-27T07:50:46.743511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743607+0000","last_change":"2024-04-27T07:50:46.743607+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743607+0000","last_undegraded":"2024-04-27T07:50:46.743607+0000","last_fullsized":"2024-04-27T07:50:46.743607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.747351+0000","last_change":"2024-04-27T07:50:46.747351+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.747351+0000","last_undegraded":"2024-04-27T07:50:46.747351+0000","last_fullsized":"2024-04-27T07:50:46.747351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745539+0000","last_change":"2024-04-27T07:50:46.745539+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745539+0000","last_undegraded":"2024-04-27T07:50:46.745539+0000","last_fullsized":"2024-04-27T07:50:46.745539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.746320+0000","last_change":"2024-04-27T07:50:46.746320+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.746320+0000","last_undegraded":"2024-04-27T07:50:46.746320+0000","last_fullsized":"2024-04-27T07:50:46.746320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":43569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":745472,"data_stored":633937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:50:52.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:50:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 Processor -- start 2024-04-27T07:50:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 -- start start 2024-04-27T07:50:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe1f0161dd0 0x7fe1f009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe1f015fa60 0x7fe1f006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f009bc00 con 0x7fe1f006dc30 2024-04-27T07:50:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f00d1b60 con 0x7fe1f015fa60 2024-04-27T07:50:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f6e4d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f009b080 con 0x7fe1f0161dd0 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe1f015fa60 0x7fe1f006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe1f015fa60 0x7fe1f006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44848/0 (socket says 172.21.15.22:44848) 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 -- 172.21.15.22:0/3856363022 learned_addr learned my addr 172.21.15.22:0/3856363022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1f4be9640 1 --2- 172.21.15.22:0/3856363022 >> v2:172.21.15.183:3300/0 conn(0x7fe1f0161dd0 0x7fe1f009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 -- 172.21.15.22:0/3856363022 >> v2:172.21.15.183:3300/0 conn(0x7fe1f0161dd0 msgr2=0x7fe1f009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 --2- 172.21.15.22:0/3856363022 >> v2:172.21.15.183:3300/0 conn(0x7fe1f0161dd0 0x7fe1f009aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 -- 172.21.15.22:0/3856363022 >> v1:172.21.15.183:6789/0 conn(0x7fe1f006dc30 legacy=0x7fe1f015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.028+0000 7fe1effff640 1 -- 172.21.15.22:0/3856363022 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f009b360 con 0x7fe1f015fa60 2024-04-27T07:50:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1effff640 1 --2- 172.21.15.22:0/3856363022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f015fa60 0x7fe1f006d6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00736e0 tx=0x7fe1e401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca10a558ed70bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1ef7fe640 1 -- 172.21.15.22:0/3856363022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40ab620 con 0x7fe1f015fa60 2024-04-27T07:50:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1ef7fe640 1 -- 172.21.15.22:0/3856363022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e40b0070 con 0x7fe1f015fa60 2024-04-27T07:50:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1ef7fe640 1 -- 172.21.15.22:0/3856363022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40ab9e0 con 0x7fe1f015fa60 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/3856363022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f015fa60 msgr2=0x7fe1f006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/3856363022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f015fa60 0x7fe1f006d6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00736e0 tx=0x7fe1e401c420 comp rx=0 tx=0).stop 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/3856363022 shutdown_connections 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/3856363022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f015fa60 0x7fe1f006d6f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/3856363022 >> v2:172.21.15.183:3300/0 conn(0x7fe1f0161dd0 0x7fe1f009aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/3856363022 >> 172.21.15.22:0/3856363022 conn(0x7fe1f00b4060 msgr2=0x7fe1f00b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/3856363022 shutdown_connections 2024-04-27T07:50:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/3856363022 wait complete. 2024-04-27T07:50:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 Processor -- start 2024-04-27T07:50:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- start start 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f006dc30 0x7fe1f0153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1f015fa60 0x7fe1f00c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01621b0 con 0x7fe1f0161dd0 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f006e030 con 0x7fe1f006dc30 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f6e4d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f0162330 con 0x7fe1f015fa60 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f53ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f53ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44850/0 (socket says 172.21.15.22:44850) 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.032+0000 7fe1f53ea640 1 -- 172.21.15.22:0/393566729 learned_addr learned my addr 172.21.15.22:0/393566729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f4be9640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f006dc30 0x7fe1f0153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1effff640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1f015fa60 0x7fe1f00c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 -- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1f015fa60 msgr2=0x7fe1f00c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1f015fa60 0x7fe1f00c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 -- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f006dc30 msgr2=0x7fe1f0153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f006dc30 0x7fe1f0153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 -- 172.21.15.22:0/393566729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f00c7040 con 0x7fe1f0161dd0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f53ea640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c7b0 tx=0x7fe1e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004b70 con 0x7fe1f0161dd0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e8004d10 con 0x7fe1f0161dd0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f00c72c0 con 0x7fe1f0161dd0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800aa90 con 0x7fe1f0161dd0 2024-04-27T07:50:53.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f00c7880 con 0x7fe1f0161dd0 2024-04-27T07:50:53.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f0068c30 con 0x7fe1f0161dd0 2024-04-27T07:50:53.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe1e800ac30 con 0x7fe1f0161dd0 2024-04-27T07:50:53.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 0x7fe1dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1e8086b80 con 0x7fe1f0161dd0 2024-04-27T07:50:53.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f4be9640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 0x7fe1dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.036+0000 7fe1f4be9640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 0x7fe1dc06a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0157910 tx=0x7fe1e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.040+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1e8020070 con 0x7fe1f0161dd0 2024-04-27T07:50:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.196+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1f0075420 con 0x7fe1dc068380 2024-04-27T07:50:53.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.212+0000 7fe1ed7fa640 1 -- 172.21.15.22:0/393566729 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7fe1f0075420 con 0x7fe1dc068380 2024-04-27T07:50:53.217 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:53.220 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:50:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 msgr2=0x7fe1dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 0x7fe1dc06a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0157910 tx=0x7fe1e0008040 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 msgr2=0x7fe1f00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c7b0 tx=0x7fe1e800cc70 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 shutdown_connections 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe1dc068380 0x7fe1dc06a840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1f0161dd0 0x7fe1f00c8ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1f015fa60 0x7fe1f00c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 --2- 172.21.15.22:0/393566729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f006dc30 0x7fe1f0153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 >> 172.21.15.22:0/393566729 conn(0x7fe1f00b4060 msgr2=0x7fe1f015acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 shutdown_connections 2024-04-27T07:50:53.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.216+0000 7fe1f6e4d640 1 -- 172.21.15.22:0/393566729 wait complete. 2024-04-27T07:50:53.235 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T07:50:52.148796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235484,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739236,"statfs":{"total":773094113280,"available":772852977664,"internally_reserved":0,"allocated":2535424,"data_stored":1590935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.399868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745982+0000","last_change":"2024-04-27T07:50:46.745982+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745982+0000","last_undegraded":"2024-04-27T07:50:46.745982+0000","last_fullsized":"2024-04-27T07:50:46.745982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750046+0000","last_change":"2024-04-27T07:50:46.750046+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750046+0000","last_undegraded":"2024-04-27T07:50:46.750046+0000","last_fullsized":"2024-04-27T07:50:46.750046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745425+0000","last_change":"2024-04-27T07:50:46.745425+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745425+0000","last_undegraded":"2024-04-27T07:50:46.745425+0000","last_fullsized":"2024-04-27T07:50:46.745425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750203+0000","last_change":"2024-04-27T07:50:46.750203+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750203+0000","last_undegraded":"2024-04-27T07:50:46.750203+0000","last_fullsized":"2024-04-27T07:50:46.750203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754693+0000","last_change":"2024-04-27T07:50:46.754693+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754693+0000","last_undegraded":"2024-04-27T07:50:46.754693+0000","last_fullsized":"2024-04-27T07:50:46.754693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751947+0000","last_change":"2024-04-27T07:50:46.751947+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751947+0000","last_undegraded":"2024-04-27T07:50:46.751947+0000","last_fullsized":"2024-04-27T07:50:46.751947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743673+0000","last_change":"2024-04-27T07:50:46.743673+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743673+0000","last_undegraded":"2024-04-27T07:50:46.743673+0000","last_fullsized":"2024-04-27T07:50:46.743673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751777+0000","last_change":"2024-04-27T07:50:46.751777+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751777+0000","last_undegraded":"2024-04-27T07:50:46.751777+0000","last_fullsized":"2024-04-27T07:50:46.751777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745246+0000","last_change":"2024-04-27T07:50:46.745246+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745246+0000","last_undegraded":"2024-04-27T07:50:46.745246+0000","last_fullsized":"2024-04-27T07:50:46.745246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751216+0000","last_change":"2024-04-27T07:50:46.751216+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751216+0000","last_undegraded":"2024-04-27T07:50:46.751216+0000","last_fullsized":"2024-04-27T07:50:46.751216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750082+0000","last_change":"2024-04-27T07:50:46.750082+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750082+0000","last_undegraded":"2024-04-27T07:50:46.750082+0000","last_fullsized":"2024-04-27T07:50:46.750082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753618+0000","last_change":"2024-04-27T07:50:46.753618+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753618+0000","last_undegraded":"2024-04-27T07:50:46.753618+0000","last_fullsized":"2024-04-27T07:50:46.753618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.748355+0000","last_change":"2024-04-27T07:50:46.748355+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.748355+0000","last_undegraded":"2024-04-27T07:50:46.748355+0000","last_fullsized":"2024-04-27T07:50:46.748355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754546+0000","last_change":"2024-04-27T07:50:46.754546+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754546+0000","last_undegraded":"2024-04-27T07:50:46.754546+0000","last_fullsized":"2024-04-27T07:50:46.754546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741421+0000","last_change":"2024-04-27T07:50:46.741421+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741421+0000","last_undegraded":"2024-04-27T07:50:46.741421+0000","last_fullsized":"2024-04-27T07:50:46.741421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753628+0000","last_change":"2024-04-27T07:50:46.753628+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753628+0000","last_undegraded":"2024-04-27T07:50:46.753628+0000","last_fullsized":"2024-04-27T07:50:46.753628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-27T07:50:46.752316+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:46.752316+0000","last_peered":"2024-04-27T07:50:46.752316+0000","last_clean":"2024-04-27T07:50:46.752316+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:46.752316+0000","last_undegraded":"2024-04-27T07:50:46.752316+0000","last_fullsized":"2024-04-27T07:50:46.752316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752306+0000","last_change":"2024-04-27T07:50:46.752306+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752306+0000","last_undegraded":"2024-04-27T07:50:46.752306+0000","last_fullsized":"2024-04-27T07:50:46.752306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743486+0000","last_change":"2024-04-27T07:50:46.743486+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743486+0000","last_undegraded":"2024-04-27T07:50:46.743486+0000","last_fullsized":"2024-04-27T07:50:46.743486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753620+0000","last_change":"2024-04-27T07:50:46.753620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753620+0000","last_undegraded":"2024-04-27T07:50:46.753620+0000","last_fullsized":"2024-04-27T07:50:46.753620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.749733+0000","last_change":"2024-04-27T07:50:46.749733+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.749733+0000","last_undegraded":"2024-04-27T07:50:46.749733+0000","last_fullsized":"2024-04-27T07:50:46.749733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741620+0000","last_change":"2024-04-27T07:50:46.741620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741620+0000","last_undegraded":"2024-04-27T07:50:46.741620+0000","last_fullsized":"2024-04-27T07:50:46.741620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743644+0000","last_change":"2024-04-27T07:50:46.743644+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743644+0000","last_undegraded":"2024-04-27T07:50:46.743644+0000","last_fullsized":"2024-04-27T07:50:46.743644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741748+0000","last_change":"2024-04-27T07:50:46.741748+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741748+0000","last_undegraded":"2024-04-27T07:50:46.741748+0000","last_fullsized":"2024-04-27T07:50:46.741748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752940+0000","last_change":"2024-04-27T07:50:46.752940+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752940+0000","last_undegraded":"2024-04-27T07:50:46.752940+0000","last_fullsized":"2024-04-27T07:50:46.752940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745623+0000","last_change":"2024-04-27T07:50:46.745623+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745623+0000","last_undegraded":"2024-04-27T07:50:46.745623+0000","last_fullsized":"2024-04-27T07:50:46.745623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741257+0000","last_change":"2024-04-27T07:50:46.741257+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741257+0000","last_undegraded":"2024-04-27T07:50:46.741257+0000","last_fullsized":"2024-04-27T07:50:46.741257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741427+0000","last_change":"2024-04-27T07:50:46.741427+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741427+0000","last_undegraded":"2024-04-27T07:50:46.741427+0000","last_fullsized":"2024-04-27T07:50:46.741427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743511+0000","last_change":"2024-04-27T07:50:46.743511+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743511+0000","last_undegraded":"2024-04-27T07:50:46.743511+0000","last_fullsized":"2024-04-27T07:50:46.743511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743607+0000","last_change":"2024-04-27T07:50:46.743607+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743607+0000","last_undegraded":"2024-04-27T07:50:46.743607+0000","last_fullsized":"2024-04-27T07:50:46.743607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.747351+0000","last_change":"2024-04-27T07:50:46.747351+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.747351+0000","last_undegraded":"2024-04-27T07:50:46.747351+0000","last_fullsized":"2024-04-27T07:50:46.747351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745539+0000","last_change":"2024-04-27T07:50:46.745539+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745539+0000","last_undegraded":"2024-04-27T07:50:46.745539+0000","last_fullsized":"2024-04-27T07:50:46.745539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.746320+0000","last_change":"2024-04-27T07:50:46.746320+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.746320+0000","last_undegraded":"2024-04-27T07:50:46.746320+0000","last_fullsized":"2024-04-27T07:50:46.746320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":43569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":745472,"data_stored":633937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:50:53.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:50:53.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.396+0000 7f769ab82640 1 Processor -- start 2024-04-27T07:50:53.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.396+0000 7f769ab82640 1 -- start start 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f76940b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f76940b5740 0x7f76940728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 0x7f7694072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f76941520d0 con 0x7f76940b5740 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f76940b3590 con 0x7f76940cea10 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f76940b3870 con 0x7f7694151cf0 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f76940b5740 0x7f76940728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f76940b5740 0x7f76940728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44856/0 (socket says 172.21.15.22:44856) 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 -- 172.21.15.22:0/479927807 learned_addr learned my addr 172.21.15.22:0/479927807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769891e640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f76940b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769911f640 1 --2- 172.21.15.22:0/479927807 >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 0x7f7694072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 -- 172.21.15.22:0/479927807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 msgr2=0x7f76940b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f76940b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 -- 172.21.15.22:0/479927807 >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 msgr2=0x7f7694072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 --2- 172.21.15.22:0/479927807 >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 0x7f7694072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 -- 172.21.15.22:0/479927807 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76940b3b50 con 0x7f76940b5740 2024-04-27T07:50:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769911f640 1 --2- 172.21.15.22:0/479927807 >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 0x7f7694072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f7693fff640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940b5740 0x7f76940728c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f769406f660 tx=0x7f768801e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8882e05deea74cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f76937fe640 1 -- 172.21.15.22:0/479927807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880ad070 con 0x7f76940b5740 2024-04-27T07:50:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f76937fe640 1 -- 172.21.15.22:0/479927807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688004230 con 0x7f76940b5740 2024-04-27T07:50:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f76937fe640 1 -- 172.21.15.22:0/479927807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880a7720 con 0x7f76940b5740 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- 172.21.15.22:0/479927807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940b5740 msgr2=0x7f76940728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940b5740 0x7f76940728c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f769406f660 tx=0x7f768801e910 comp rx=0 tx=0).stop 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- 172.21.15.22:0/479927807 shutdown_connections 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- 172.21.15.22:0/479927807 >> v2:172.21.15.183:3300/0 conn(0x7f76940cea10 0x7f7694072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940b5740 0x7f76940728c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 --2- 172.21.15.22:0/479927807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f76940b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- 172.21.15.22:0/479927807 >> 172.21.15.22:0/479927807 conn(0x7f7694067f10 msgr2=0x7f769406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- 172.21.15.22:0/479927807 shutdown_connections 2024-04-27T07:50:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.400+0000 7f769ab82640 1 -- 172.21.15.22:0/479927807 wait complete. 2024-04-27T07:50:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 Processor -- start 2024-04-27T07:50:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- start start 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 0x7f7694159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f769415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f769415e9a0 con 0x7f76940cea10 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f769415e820 con 0x7f76940b5740 2024-04-27T07:50:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f769415eb20 con 0x7f7694151cf0 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44858/0 (socket says 172.21.15.22:44858) 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 -- 172.21.15.22:0/1562036825 learned_addr learned my addr 172.21.15.22:0/1562036825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769891e640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 0x7f7694159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769911f640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f769415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 -- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 msgr2=0x7f769415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f769415d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 -- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 msgr2=0x7f7694159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 0x7f7694159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 -- 172.21.15.22:0/1562036825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7694158dc0 con 0x7f76940cea10 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769891e640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 0x7f7694159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f7693fff640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f768801e740 tx=0x7f768801b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880be070 con 0x7f76940cea10 2024-04-27T07:50:53.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688004030 con 0x7f76940cea10 2024-04-27T07:50:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769415de80 con 0x7f76940cea10 2024-04-27T07:50:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880ad040 con 0x7f76940cea10 2024-04-27T07:50:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7694159080 con 0x7f76940cea10 2024-04-27T07:50:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7694068c30 con 0x7f76940cea10 2024-04-27T07:50:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.404+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f76880c1070 con 0x7f76940cea10 2024-04-27T07:50:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.408+0000 7f76917fa640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 0x7f768006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.408+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7688120890 con 0x7f76940cea10 2024-04-27T07:50:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.408+0000 7f769891e640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 0x7f768006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.408+0000 7f769891e640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 0x7f768006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7684006fd0 tx=0x7f7684008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.412+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76880185c0 con 0x7f76940cea10 2024-04-27T07:50:53.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.568+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76940af5f0 con 0x7f7680068380 2024-04-27T07:50:53.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.584+0000 7f76917fa640 1 -- 172.21.15.22:0/1562036825 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7f76940af5f0 con 0x7f7680068380 2024-04-27T07:50:53.591 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:53.593 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 msgr2=0x7f768006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 0x7f768006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7684006fd0 tx=0x7f7684008040 comp rx=0 tx=0).stop 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 msgr2=0x7f769415cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f768801e740 tx=0x7f768801b6b0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 shutdown_connections 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7680068380 0x7f768006a840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7694151cf0 0x7f769415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76940cea10 0x7f769415cf00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 --2- 172.21.15.22:0/1562036825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76940b5740 0x7f7694159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 >> 172.21.15.22:0/1562036825 conn(0x7f7694067f10 msgr2=0x7f76940728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:53.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 shutdown_connections 2024-04-27T07:50:53.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.588+0000 7f769ab82640 1 -- 172.21.15.22:0/1562036825 wait complete. 2024-04-27T07:50:53.609 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T07:50:52.148796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235484,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739236,"statfs":{"total":773094113280,"available":772852977664,"internally_reserved":0,"allocated":2535424,"data_stored":1590935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.399868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745982+0000","last_change":"2024-04-27T07:50:46.745982+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745982+0000","last_undegraded":"2024-04-27T07:50:46.745982+0000","last_fullsized":"2024-04-27T07:50:46.745982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750046+0000","last_change":"2024-04-27T07:50:46.750046+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750046+0000","last_undegraded":"2024-04-27T07:50:46.750046+0000","last_fullsized":"2024-04-27T07:50:46.750046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745425+0000","last_change":"2024-04-27T07:50:46.745425+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745425+0000","last_undegraded":"2024-04-27T07:50:46.745425+0000","last_fullsized":"2024-04-27T07:50:46.745425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750203+0000","last_change":"2024-04-27T07:50:46.750203+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750203+0000","last_undegraded":"2024-04-27T07:50:46.750203+0000","last_fullsized":"2024-04-27T07:50:46.750203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754693+0000","last_change":"2024-04-27T07:50:46.754693+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754693+0000","last_undegraded":"2024-04-27T07:50:46.754693+0000","last_fullsized":"2024-04-27T07:50:46.754693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751947+0000","last_change":"2024-04-27T07:50:46.751947+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751947+0000","last_undegraded":"2024-04-27T07:50:46.751947+0000","last_fullsized":"2024-04-27T07:50:46.751947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743673+0000","last_change":"2024-04-27T07:50:46.743673+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743673+0000","last_undegraded":"2024-04-27T07:50:46.743673+0000","last_fullsized":"2024-04-27T07:50:46.743673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751777+0000","last_change":"2024-04-27T07:50:46.751777+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751777+0000","last_undegraded":"2024-04-27T07:50:46.751777+0000","last_fullsized":"2024-04-27T07:50:46.751777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745246+0000","last_change":"2024-04-27T07:50:46.745246+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745246+0000","last_undegraded":"2024-04-27T07:50:46.745246+0000","last_fullsized":"2024-04-27T07:50:46.745246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.751216+0000","last_change":"2024-04-27T07:50:46.751216+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.751216+0000","last_undegraded":"2024-04-27T07:50:46.751216+0000","last_fullsized":"2024-04-27T07:50:46.751216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.750082+0000","last_change":"2024-04-27T07:50:46.750082+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.750082+0000","last_undegraded":"2024-04-27T07:50:46.750082+0000","last_fullsized":"2024-04-27T07:50:46.750082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753618+0000","last_change":"2024-04-27T07:50:46.753618+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753618+0000","last_undegraded":"2024-04-27T07:50:46.753618+0000","last_fullsized":"2024-04-27T07:50:46.753618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.748355+0000","last_change":"2024-04-27T07:50:46.748355+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.748355+0000","last_undegraded":"2024-04-27T07:50:46.748355+0000","last_fullsized":"2024-04-27T07:50:46.748355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.754546+0000","last_change":"2024-04-27T07:50:46.754546+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.754546+0000","last_undegraded":"2024-04-27T07:50:46.754546+0000","last_fullsized":"2024-04-27T07:50:46.754546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741421+0000","last_change":"2024-04-27T07:50:46.741421+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741421+0000","last_undegraded":"2024-04-27T07:50:46.741421+0000","last_fullsized":"2024-04-27T07:50:46.741421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753628+0000","last_change":"2024-04-27T07:50:46.753628+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753628+0000","last_undegraded":"2024-04-27T07:50:46.753628+0000","last_fullsized":"2024-04-27T07:50:46.753628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-27T07:50:46.752316+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:46.752316+0000","last_peered":"2024-04-27T07:50:46.752316+0000","last_clean":"2024-04-27T07:50:46.752316+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:46.752316+0000","last_undegraded":"2024-04-27T07:50:46.752316+0000","last_fullsized":"2024-04-27T07:50:46.752316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752306+0000","last_change":"2024-04-27T07:50:46.752306+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752306+0000","last_undegraded":"2024-04-27T07:50:46.752306+0000","last_fullsized":"2024-04-27T07:50:46.752306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743486+0000","last_change":"2024-04-27T07:50:46.743486+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743486+0000","last_undegraded":"2024-04-27T07:50:46.743486+0000","last_fullsized":"2024-04-27T07:50:46.743486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.753620+0000","last_change":"2024-04-27T07:50:46.753620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.753620+0000","last_undegraded":"2024-04-27T07:50:46.753620+0000","last_fullsized":"2024-04-27T07:50:46.753620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.749733+0000","last_change":"2024-04-27T07:50:46.749733+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.749733+0000","last_undegraded":"2024-04-27T07:50:46.749733+0000","last_fullsized":"2024-04-27T07:50:46.749733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741620+0000","last_change":"2024-04-27T07:50:46.741620+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741620+0000","last_undegraded":"2024-04-27T07:50:46.741620+0000","last_fullsized":"2024-04-27T07:50:46.741620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743644+0000","last_change":"2024-04-27T07:50:46.743644+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743644+0000","last_undegraded":"2024-04-27T07:50:46.743644+0000","last_fullsized":"2024-04-27T07:50:46.743644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741748+0000","last_change":"2024-04-27T07:50:46.741748+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741748+0000","last_undegraded":"2024-04-27T07:50:46.741748+0000","last_fullsized":"2024-04-27T07:50:46.741748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.752940+0000","last_change":"2024-04-27T07:50:46.752940+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.752940+0000","last_undegraded":"2024-04-27T07:50:46.752940+0000","last_fullsized":"2024-04-27T07:50:46.752940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745623+0000","last_change":"2024-04-27T07:50:46.745623+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745623+0000","last_undegraded":"2024-04-27T07:50:46.745623+0000","last_fullsized":"2024-04-27T07:50:46.745623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741257+0000","last_change":"2024-04-27T07:50:46.741257+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741257+0000","last_undegraded":"2024-04-27T07:50:46.741257+0000","last_fullsized":"2024-04-27T07:50:46.741257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.741427+0000","last_change":"2024-04-27T07:50:46.741427+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.741427+0000","last_undegraded":"2024-04-27T07:50:46.741427+0000","last_fullsized":"2024-04-27T07:50:46.741427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743511+0000","last_change":"2024-04-27T07:50:46.743511+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743511+0000","last_undegraded":"2024-04-27T07:50:46.743511+0000","last_fullsized":"2024-04-27T07:50:46.743511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.743607+0000","last_change":"2024-04-27T07:50:46.743607+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.743607+0000","last_undegraded":"2024-04-27T07:50:46.743607+0000","last_fullsized":"2024-04-27T07:50:46.743607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.747351+0000","last_change":"2024-04-27T07:50:46.747351+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.747351+0000","last_undegraded":"2024-04-27T07:50:46.747351+0000","last_fullsized":"2024-04-27T07:50:46.747351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.745539+0000","last_change":"2024-04-27T07:50:46.745539+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.745539+0000","last_undegraded":"2024-04-27T07:50:46.745539+0000","last_fullsized":"2024-04-27T07:50:46.745539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:50:47.736491+0000","last_change":"2024-04-27T07:50:47.736491+0000","last_active":"2024-04-27T07:50:47.736491+0000","last_peered":"2024-04-27T07:50:47.736491+0000","last_clean":"2024-04-27T07:50:47.736491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:47.736491+0000","last_undegraded":"2024-04-27T07:50:47.736491+0000","last_fullsized":"2024-04-27T07:50:47.736491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-27T07:50:46.746320+0000","last_change":"2024-04-27T07:50:46.746320+0000","last_active":"2024-04-27T07:50:46.727961+0000","last_peered":"2024-04-27T07:50:46.727961+0000","last_clean":"2024-04-27T07:50:46.727961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:50:46.746320+0000","last_undegraded":"2024-04-27T07:50:46.746320+0000","last_fullsized":"2024-04-27T07:50:46.746320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":43569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":745472,"data_stored":633937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-04-27T07:50:53.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-04-27T07:50:53.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-27T07:50:53.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-04-27T07:50:53.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-04-27T07:50:53.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-04-27T07:50:53.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.745982+0000', 'last_change': '2024-04-27T07:50:46.745982+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.745982+0000', 'last_undegraded': '2024-04-27T07:50:46.745982+0000', 'last_fullsized': '2024-04-27T07:50:46.745982+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-27T07:50:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-04-27T07:50:53.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.750046+0000', 'last_change': '2024-04-27T07:50:46.750046+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.750046+0000', 'last_undegraded': '2024-04-27T07:50:46.750046+0000', 'last_fullsized': '2024-04-27T07:50:46.750046+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:50:53.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-04-27T07:50:53.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.745425+0000', 'last_change': '2024-04-27T07:50:46.745425+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.745425+0000', 'last_undegraded': '2024-04-27T07:50:46.745425+0000', 'last_fullsized': '2024-04-27T07:50:46.745425+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:50:53.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-27T07:50:53.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-04-27T07:50:53.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.750203+0000', 'last_change': '2024-04-27T07:50:46.750203+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.750203+0000', 'last_undegraded': '2024-04-27T07:50:46.750203+0000', 'last_fullsized': '2024-04-27T07:50:46.750203+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:50:53.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-27T07:50:53.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.754693+0000', 'last_change': '2024-04-27T07:50:46.754693+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.754693+0000', 'last_undegraded': '2024-04-27T07:50:46.754693+0000', 'last_fullsized': '2024-04-27T07:50:46.754693+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.751947+0000', 'last_change': '2024-04-27T07:50:46.751947+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.751947+0000', 'last_undegraded': '2024-04-27T07:50:46.751947+0000', 'last_fullsized': '2024-04-27T07:50:46.751947+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-27T07:50:53.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.743673+0000', 'last_change': '2024-04-27T07:50:46.743673+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.743673+0000', 'last_undegraded': '2024-04-27T07:50:46.743673+0000', 'last_fullsized': '2024-04-27T07:50:46.743673+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-04-27T07:50:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.751777+0000', 'last_change': '2024-04-27T07:50:46.751777+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.751777+0000', 'last_undegraded': '2024-04-27T07:50:46.751777+0000', 'last_fullsized': '2024-04-27T07:50:46.751777+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.745246+0000', 'last_change': '2024-04-27T07:50:46.745246+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.745246+0000', 'last_undegraded': '2024-04-27T07:50:46.745246+0000', 'last_fullsized': '2024-04-27T07:50:46.745246+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-04-27T07:50:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.751216+0000', 'last_change': '2024-04-27T07:50:46.751216+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.751216+0000', 'last_undegraded': '2024-04-27T07:50:46.751216+0000', 'last_fullsized': '2024-04-27T07:50:46.751216+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.750082+0000', 'last_change': '2024-04-27T07:50:46.750082+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.750082+0000', 'last_undegraded': '2024-04-27T07:50:46.750082+0000', 'last_fullsized': '2024-04-27T07:50:46.750082+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-04-27T07:50:53.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.753618+0000', 'last_change': '2024-04-27T07:50:46.753618+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.753618+0000', 'last_undegraded': '2024-04-27T07:50:46.753618+0000', 'last_fullsized': '2024-04-27T07:50:46.753618+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-04-27T07:50:53.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.748355+0000', 'last_change': '2024-04-27T07:50:46.748355+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.748355+0000', 'last_undegraded': '2024-04-27T07:50:46.748355+0000', 'last_fullsized': '2024-04-27T07:50:46.748355+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.754546+0000', 'last_change': '2024-04-27T07:50:46.754546+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.754546+0000', 'last_undegraded': '2024-04-27T07:50:46.754546+0000', 'last_fullsized': '2024-04-27T07:50:46.754546+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-04-27T07:50:53.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.741421+0000', 'last_change': '2024-04-27T07:50:46.741421+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.741421+0000', 'last_undegraded': '2024-04-27T07:50:46.741421+0000', 'last_fullsized': '2024-04-27T07:50:46.741421+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.753628+0000', 'last_change': '2024-04-27T07:50:46.753628+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.753628+0000', 'last_undegraded': '2024-04-27T07:50:46.753628+0000', 'last_fullsized': '2024-04-27T07:50:46.753628+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-04-27T07:50:53.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.752306+0000', 'last_change': '2024-04-27T07:50:46.752306+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.752306+0000', 'last_undegraded': '2024-04-27T07:50:46.752306+0000', 'last_fullsized': '2024-04-27T07:50:46.752306+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.743486+0000', 'last_change': '2024-04-27T07:50:46.743486+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.743486+0000', 'last_undegraded': '2024-04-27T07:50:46.743486+0000', 'last_fullsized': '2024-04-27T07:50:46.743486+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-04-27T07:50:53.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.753620+0000', 'last_change': '2024-04-27T07:50:46.753620+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.753620+0000', 'last_undegraded': '2024-04-27T07:50:46.753620+0000', 'last_fullsized': '2024-04-27T07:50:46.753620+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.749733+0000', 'last_change': '2024-04-27T07:50:46.749733+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.749733+0000', 'last_undegraded': '2024-04-27T07:50:46.749733+0000', 'last_fullsized': '2024-04-27T07:50:46.749733+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.741620+0000', 'last_change': '2024-04-27T07:50:46.741620+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.741620+0000', 'last_undegraded': '2024-04-27T07:50:46.741620+0000', 'last_fullsized': '2024-04-27T07:50:46.741620+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-27T07:50:53.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.743644+0000', 'last_change': '2024-04-27T07:50:46.743644+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.743644+0000', 'last_undegraded': '2024-04-27T07:50:46.743644+0000', 'last_fullsized': '2024-04-27T07:50:46.743644+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.741748+0000', 'last_change': '2024-04-27T07:50:46.741748+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.741748+0000', 'last_undegraded': '2024-04-27T07:50:46.741748+0000', 'last_fullsized': '2024-04-27T07:50:46.741748+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-27T07:50:53.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.752940+0000', 'last_change': '2024-04-27T07:50:46.752940+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.752940+0000', 'last_undegraded': '2024-04-27T07:50:46.752940+0000', 'last_fullsized': '2024-04-27T07:50:46.752940+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.745623+0000', 'last_change': '2024-04-27T07:50:46.745623+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.745623+0000', 'last_undegraded': '2024-04-27T07:50:46.745623+0000', 'last_fullsized': '2024-04-27T07:50:46.745623+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.741257+0000', 'last_change': '2024-04-27T07:50:46.741257+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.741257+0000', 'last_undegraded': '2024-04-27T07:50:46.741257+0000', 'last_fullsized': '2024-04-27T07:50:46.741257+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.741427+0000', 'last_change': '2024-04-27T07:50:46.741427+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.741427+0000', 'last_undegraded': '2024-04-27T07:50:46.741427+0000', 'last_fullsized': '2024-04-27T07:50:46.741427+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.743511+0000', 'last_change': '2024-04-27T07:50:46.743511+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.743511+0000', 'last_undegraded': '2024-04-27T07:50:46.743511+0000', 'last_fullsized': '2024-04-27T07:50:46.743511+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-27T07:50:53.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.743607+0000', 'last_change': '2024-04-27T07:50:46.743607+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.743607+0000', 'last_undegraded': '2024-04-27T07:50:46.743607+0000', 'last_fullsized': '2024-04-27T07:50:46.743607+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.747351+0000', 'last_change': '2024-04-27T07:50:46.747351+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.747351+0000', 'last_undegraded': '2024-04-27T07:50:46.747351+0000', 'last_fullsized': '2024-04-27T07:50:46.747351+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-04-27T07:50:53.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.745539+0000', 'last_change': '2024-04-27T07:50:46.745539+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.745539+0000', 'last_undegraded': '2024-04-27T07:50:46.745539+0000', 'last_fullsized': '2024-04-27T07:50:46.745539+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:50:53.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-27T07:50:53.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:50:47.736491+0000', 'last_change': '2024-04-27T07:50:47.736491+0000', 'last_active': '2024-04-27T07:50:47.736491+0000', 'last_peered': '2024-04-27T07:50:47.736491+0000', 'last_clean': '2024-04-27T07:50:47.736491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:47.736491+0000', 'last_undegraded': '2024-04-27T07:50:47.736491+0000', 'last_fullsized': '2024-04-27T07:50:47.736491+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:47.736491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:50:53.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-04-27T07:50:53.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:50:46.746320+0000', 'last_change': '2024-04-27T07:50:46.746320+0000', 'last_active': '2024-04-27T07:50:46.727961+0000', 'last_peered': '2024-04-27T07:50:46.727961+0000', 'last_clean': '2024-04-27T07:50:46.727961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:50:46.746320+0000', 'last_undegraded': '2024-04-27T07:50:46.746320+0000', 'last_fullsized': '2024-04-27T07:50:46.746320+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'last_clean_scrub_stamp': '2024-04-27T07:50:46.727961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-27T07:50:53.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-27T07:50:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.756+0000 7fd828847640 1 Processor -- start 2024-04-27T07:50:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.756+0000 7fd828847640 1 -- start start 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.756+0000 7fd828847640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd8240cceb0 0x7fd8240cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.756+0000 7fd828847640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd824071f90 0x7fd824072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.756+0000 7fd828847640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8240cd900 con 0x7fd824145ee0 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd8240cdbe0 con 0x7fd8240cceb0 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd8240cdec0 con 0x7fd824071f90 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd8240cceb0 0x7fd8240cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd8240cceb0 0x7fd8240cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44866/0 (socket says 172.21.15.22:44866) 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 -- 172.21.15.22:0/2643295700 learned_addr learned my addr 172.21.15.22:0/2643295700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd822d9d640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd824071f90 0x7fd824072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 -- 172.21.15.22:0/2643295700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd824071f90 msgr2=0x7fd824072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd824071f90 0x7fd824072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 -- 172.21.15.22:0/2643295700 >> v1:172.21.15.183:6789/0 conn(0x7fd824145ee0 legacy=0x7fd8240ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 -- 172.21.15.22:0/2643295700 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8240ce1a0 con 0x7fd8240cceb0 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd821d9b640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8240cceb0 0x7fd8240cd2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f667083af30260f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd82159a640 1 -- 172.21.15.22:0/2643295700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8180b5070 con 0x7fd8240cceb0 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd82159a640 1 -- 172.21.15.22:0/2643295700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8180a8a50 con 0x7fd8240cceb0 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd82159a640 1 -- 172.21.15.22:0/2643295700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8180a8e20 con 0x7fd8240cceb0 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- 172.21.15.22:0/2643295700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8240cceb0 msgr2=0x7fd8240cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8240cceb0 0x7fd8240cd2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81801ed50 comp rx=0 tx=0).stop 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- 172.21.15.22:0/2643295700 shutdown_connections 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd824071f90 0x7fd824072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 --2- 172.21.15.22:0/2643295700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8240cceb0 0x7fd8240cd2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- 172.21.15.22:0/2643295700 >> 172.21.15.22:0/2643295700 conn(0x7fd82406bbd0 msgr2=0x7fd82406bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:53.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- 172.21.15.22:0/2643295700 shutdown_connections 2024-04-27T07:50:53.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.760+0000 7fd828847640 1 -- 172.21.15.22:0/2643295700 wait complete. 2024-04-27T07:50:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 Processor -- start 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- start start 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 0x7fd8241f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8240cceb0 0x7fd82413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd824160500 con 0x7fd824145ee0 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd824160380 con 0x7fd824071f90 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd82415cba0 con 0x7fd8240cceb0 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44882/0 (socket says 172.21.15.22:44882) 2024-04-27T07:50:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 -- 172.21.15.22:0/1915612158 learned_addr learned my addr 172.21.15.22:0/1915612158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 -- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8240cceb0 msgr2=0x7fd82413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8240cceb0 0x7fd82413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 -- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 msgr2=0x7fd8241f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd82259c640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 0x7fd8241f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 0x7fd8241f2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 -- 172.21.15.22:0/1915612158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82413dd10 con 0x7fd824145ee0 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd82259c640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 0x7fd8241f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:53.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd822d9d640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd81c007d90 tx=0x7fd81c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c014420 con 0x7fd824145ee0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c019070 con 0x7fd824145ee0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c014820 con 0x7fd824145ee0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82413dff0 con 0x7fd824145ee0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82413e5b0 con 0x7fd824145ee0 2024-04-27T07:50:53.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd824068c30 con 0x7fd824145ee0 2024-04-27T07:50:53.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.764+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd81c0149c0 con 0x7fd824145ee0 2024-04-27T07:50:53.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.768+0000 7fd812ffd640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 0x7fd80406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:53.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.768+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd81c087230 con 0x7fd824145ee0 2024-04-27T07:50:53.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.768+0000 7fd82259c640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 0x7fd80406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:53.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.768+0000 7fd82259c640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 0x7fd80406a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd814001650 tx=0x7fd814009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:53.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.772+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd81c0577f0 con 0x7fd824145ee0 2024-04-27T07:50:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.992+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd82406f640 con 0x7fd824145ee0 2024-04-27T07:50:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.992+0000 7fd812ffd640 1 -- 172.21.15.22:0/1915612158 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1107 (secure 0 0 0) 0x7fd81c0570f0 con 0x7fd824145ee0 2024-04-27T07:50:53.997 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:53.998 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":22,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1714204241,"num_in_osds":8,"osd_in_since":1714204233,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241135616,"bytes_avail":772852977664,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4273}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-27T07:50:31.352005+0000","services":{}},"progress_events":{}} 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 msgr2=0x7fd80406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 0x7fd80406a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd814001650 tx=0x7fd814009040 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 msgr2=0x7fd8241f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd81c007d90 tx=0x7fd81c00e4b0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 shutdown_connections 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd804068380 0x7fd80406a840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd824145ee0 0x7fd8241f5a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8240cceb0 0x7fd82413cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 --2- 172.21.15.22:0/1915612158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824071f90 0x7fd8241f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 >> 172.21.15.22:0/1915612158 conn(0x7fd82406bbd0 msgr2=0x7fd824072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 shutdown_connections 2024-04-27T07:50:54.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:53.996+0000 7fd828847640 1 -- 172.21.15.22:0/1915612158 wait complete. 2024-04-27T07:50:54.015 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-27T07:50:54.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 Processor -- start 2024-04-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 -- start start 2024-04-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7effcc1454b0 0x7effcc072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effcc06e050 con 0x7effcc145a00 2024-04-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7effcc06f330 con 0x7effcc1454b0 2024-04-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd2a29640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7effcc0ceae0 con 0x7effcc072740 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:52530/0 (socket says 172.21.15.22:52530) 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 -- 172.21.15.22:0/1904697371 learned_addr learned my addr 172.21.15.22:0/1904697371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effcbfff640 1 --2- 172.21.15.22:0/1904697371 >> v2:172.21.15.183:3300/0 conn(0x7effcc1454b0 0x7effcc072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 -- 172.21.15.22:0/1904697371 >> v2:172.21.15.183:3300/0 conn(0x7effcc1454b0 msgr2=0x7effcc072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 --2- 172.21.15.22:0/1904697371 >> v2:172.21.15.183:3300/0 conn(0x7effcc1454b0 0x7effcc072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 -- 172.21.15.22:0/1904697371 >> v1:172.21.15.183:6789/0 conn(0x7effcc145a00 legacy=0x7effcc072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.132+0000 7effd0fc6640 1 -- 172.21.15.22:0/1904697371 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc0cedc0 con 0x7effcc072740 2024-04-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd0fc6640 1 --2- 172.21.15.22:0/1904697371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effc4009e50 tx=0x7effc401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee8b237227762e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effcaffd640 1 -- 172.21.15.22:0/1904697371 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc40a5700 con 0x7effcc072740 2024-04-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effcaffd640 1 -- 172.21.15.22:0/1904697371 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc40a58a0 con 0x7effcc072740 2024-04-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effcaffd640 1 -- 172.21.15.22:0/1904697371 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc40a5c70 con 0x7effcc072740 2024-04-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- 172.21.15.22:0/1904697371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 msgr2=0x7effcc06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 --2- 172.21.15.22:0/1904697371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effc4009e50 tx=0x7effc401ca30 comp rx=0 tx=0).stop 2024-04-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- 172.21.15.22:0/1904697371 shutdown_connections 2024-04-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 --2- 172.21.15.22:0/1904697371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc072740 0x7effcc06edf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 --2- 172.21.15.22:0/1904697371 >> v2:172.21.15.183:3300/0 conn(0x7effcc1454b0 0x7effcc072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- 172.21.15.22:0/1904697371 >> 172.21.15.22:0/1904697371 conn(0x7effcc067f10 msgr2=0x7effcc06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- 172.21.15.22:0/1904697371 shutdown_connections 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- 172.21.15.22:0/1904697371 wait complete. 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 Processor -- start 2024-04-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.136+0000 7effd2a29640 1 -- start start 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effcc1454b0 0x7effcc137d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc145a00 0x7effcc13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7effcc15d690 con 0x7effcc072740 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effcc15d510 con 0x7effcc1454b0 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7effcc15d810 con 0x7effcc145a00 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44894/0 (socket says 172.21.15.22:44894) 2024-04-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 -- 172.21.15.22:0/3208605534 learned_addr learned my addr 172.21.15.22:0/3208605534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd0fc6640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc145a00 0x7effcc13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcb7fe640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effcc1454b0 0x7effcc137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 -- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc145a00 msgr2=0x7effcc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc145a00 0x7effcc13f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 -- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effcc1454b0 msgr2=0x7effcc137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effcc1454b0 0x7effcc137d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 -- 172.21.15.22:0/3208605534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc138240 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcbfff640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effc000e960 tx=0x7effc000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0017070 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc13e990 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc15a6e0 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc000ca00 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000cdb0 con 0x7effcc072740 2024-04-27T07:50:54.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effac0052a0 con 0x7effcc072740 2024-04-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7effc001a070 con 0x7effcc072740 2024-04-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 0x7effb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7effc0086a00 con 0x7effcc072740 2024-04-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.140+0000 7effcb7fe640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 0x7effb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.144+0000 7effcb7fe640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 0x7effb806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effbc007fd0 tx=0x7effbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:54.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.144+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effc0056fc0 con 0x7effcc072740 2024-04-27T07:50:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.304+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effac000bd0 con 0x7effb8068380 2024-04-27T07:50:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.320+0000 7effc8ff9640 1 -- 172.21.15.22:0/3208605534 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7effac000bd0 con 0x7effb8068380 2024-04-27T07:50:54.328 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:50:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 msgr2=0x7effb806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 0x7effb806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effbc007fd0 tx=0x7effbc009040 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 msgr2=0x7effcc15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effc000e960 tx=0x7effc000ee20 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 shutdown_connections 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effb8068380 0x7effb806a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effcc145a00 0x7effcc13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effcc1454b0 0x7effcc137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 --2- 172.21.15.22:0/3208605534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effcc072740 0x7effcc15b1c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 >> 172.21.15.22:0/3208605534 conn(0x7effcc067f10 msgr2=0x7effcc143700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 shutdown_connections 2024-04-27T07:50:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:54.324+0000 7effd2a29640 1 -- 172.21.15.22:0/3208605534 wait complete. 2024-04-27T07:50:54.345 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-27T07:50:54.149354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235696,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739024,"statfs":{"total":773094113280,"available":772852760576,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.400426"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283542+0000","last_change":"2024-04-27T07:50:48.807748+0000","last_active":"2024-04-27T07:50:51.283542+0000","last_peered":"2024-04-27T07:50:51.283542+0000","last_clean":"2024-04-27T07:50:51.283542+0000","last_became_active":"2024-04-27T07:50:48.806582+0000","last_became_peered":"2024-04-27T07:50:48.806582+0000","last_unstale":"2024-04-27T07:50:51.283542+0000","last_undegraded":"2024-04-27T07:50:51.283542+0000","last_fullsized":"2024-04-27T07:50:51.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.223671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282205+0000","last_change":"2024-04-27T07:50:48.809136+0000","last_active":"2024-04-27T07:50:51.282205+0000","last_peered":"2024-04-27T07:50:51.282205+0000","last_clean":"2024-04-27T07:50:51.282205+0000","last_became_active":"2024-04-27T07:50:48.808581+0000","last_became_peered":"2024-04-27T07:50:48.808581+0000","last_unstale":"2024-04-27T07:50:51.282205+0000","last_undegraded":"2024-04-27T07:50:51.282205+0000","last_fullsized":"2024-04-27T07:50:51.282205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923394+0000","last_change":"2024-04-27T07:50:48.803232+0000","last_active":"2024-04-27T07:50:50.923394+0000","last_peered":"2024-04-27T07:50:50.923394+0000","last_clean":"2024-04-27T07:50:50.923394+0000","last_became_active":"2024-04-27T07:50:48.802828+0000","last_became_peered":"2024-04-27T07:50:48.802828+0000","last_unstale":"2024-04-27T07:50:50.923394+0000","last_undegraded":"2024-04-27T07:50:50.923394+0000","last_fullsized":"2024-04-27T07:50:50.923394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800701+0000","last_change":"2024-04-27T07:50:48.822942+0000","last_active":"2024-04-27T07:50:49.800701+0000","last_peered":"2024-04-27T07:50:49.800701+0000","last_clean":"2024-04-27T07:50:49.800701+0000","last_became_active":"2024-04-27T07:50:48.822709+0000","last_became_peered":"2024-04-27T07:50:48.822709+0000","last_unstale":"2024-04-27T07:50:49.800701+0000","last_undegraded":"2024-04-27T07:50:49.800701+0000","last_fullsized":"2024-04-27T07:50:49.800701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:38.768492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283631+0000","last_change":"2024-04-27T07:50:48.805852+0000","last_active":"2024-04-27T07:50:51.283631+0000","last_peered":"2024-04-27T07:50:51.283631+0000","last_clean":"2024-04-27T07:50:51.283631+0000","last_became_active":"2024-04-27T07:50:48.805553+0000","last_became_peered":"2024-04-27T07:50:48.805553+0000","last_unstale":"2024-04-27T07:50:51.283631+0000","last_undegraded":"2024-04-27T07:50:51.283631+0000","last_fullsized":"2024-04-27T07:50:51.283631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:19:06.682886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800634+0000","last_change":"2024-04-27T07:50:48.795482+0000","last_active":"2024-04-27T07:50:49.800634+0000","last_peered":"2024-04-27T07:50:49.800634+0000","last_clean":"2024-04-27T07:50:49.800634+0000","last_became_active":"2024-04-27T07:50:48.793813+0000","last_became_peered":"2024-04-27T07:50:48.793813+0000","last_unstale":"2024-04-27T07:50:49.800634+0000","last_undegraded":"2024-04-27T07:50:49.800634+0000","last_fullsized":"2024-04-27T07:50:49.800634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:39:48.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282786+0000","last_change":"2024-04-27T07:50:48.821409+0000","last_active":"2024-04-27T07:50:51.282786+0000","last_peered":"2024-04-27T07:50:51.282786+0000","last_clean":"2024-04-27T07:50:51.282786+0000","last_became_active":"2024-04-27T07:50:48.821162+0000","last_became_peered":"2024-04-27T07:50:48.821162+0000","last_unstale":"2024-04-27T07:50:51.282786+0000","last_undegraded":"2024-04-27T07:50:51.282786+0000","last_fullsized":"2024-04-27T07:50:51.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:20:43.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281085+0000","last_change":"2024-04-27T07:50:48.830026+0000","last_active":"2024-04-27T07:50:51.281085+0000","last_peered":"2024-04-27T07:50:51.281085+0000","last_clean":"2024-04-27T07:50:51.281085+0000","last_became_active":"2024-04-27T07:50:48.821303+0000","last_became_peered":"2024-04-27T07:50:48.821303+0000","last_unstale":"2024-04-27T07:50:51.281085+0000","last_undegraded":"2024-04-27T07:50:51.281085+0000","last_fullsized":"2024-04-27T07:50:51.281085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283709+0000","last_change":"2024-04-27T07:50:48.820925+0000","last_active":"2024-04-27T07:50:51.283709+0000","last_peered":"2024-04-27T07:50:51.283709+0000","last_clean":"2024-04-27T07:50:51.283709+0000","last_became_active":"2024-04-27T07:50:48.820575+0000","last_became_peered":"2024-04-27T07:50:48.820575+0000","last_unstale":"2024-04-27T07:50:51.283709+0000","last_undegraded":"2024-04-27T07:50:51.283709+0000","last_fullsized":"2024-04-27T07:50:51.283709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:40.499571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446110+0000","last_change":"2024-04-27T07:50:48.814093+0000","last_active":"2024-04-27T07:50:50.446110+0000","last_peered":"2024-04-27T07:50:50.446110+0000","last_clean":"2024-04-27T07:50:50.446110+0000","last_became_active":"2024-04-27T07:50:48.813643+0000","last_became_peered":"2024-04-27T07:50:48.813643+0000","last_unstale":"2024-04-27T07:50:50.446110+0000","last_undegraded":"2024-04-27T07:50:50.446110+0000","last_fullsized":"2024-04-27T07:50:50.446110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:24.403447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445905+0000","last_change":"2024-04-27T07:50:48.815381+0000","last_active":"2024-04-27T07:50:50.445905+0000","last_peered":"2024-04-27T07:50:50.445905+0000","last_clean":"2024-04-27T07:50:50.445905+0000","last_became_active":"2024-04-27T07:50:48.814574+0000","last_became_peered":"2024-04-27T07:50:48.814574+0000","last_unstale":"2024-04-27T07:50:50.445905+0000","last_undegraded":"2024-04-27T07:50:50.445905+0000","last_fullsized":"2024-04-27T07:50:50.445905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:25.162778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800227+0000","last_change":"2024-04-27T07:50:48.806044+0000","last_active":"2024-04-27T07:50:49.800227+0000","last_peered":"2024-04-27T07:50:49.800227+0000","last_clean":"2024-04-27T07:50:49.800227+0000","last_became_active":"2024-04-27T07:50:48.805705+0000","last_became_peered":"2024-04-27T07:50:48.805705+0000","last_unstale":"2024-04-27T07:50:49.800227+0000","last_undegraded":"2024-04-27T07:50:49.800227+0000","last_fullsized":"2024-04-27T07:50:49.800227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:21.183002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282310+0000","last_change":"2024-04-27T07:50:48.819990+0000","last_active":"2024-04-27T07:50:51.282310+0000","last_peered":"2024-04-27T07:50:51.282310+0000","last_clean":"2024-04-27T07:50:51.282310+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.282310+0000","last_undegraded":"2024-04-27T07:50:51.282310+0000","last_fullsized":"2024-04-27T07:50:51.282310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922720+0000","last_change":"2024-04-27T07:50:48.804618+0000","last_active":"2024-04-27T07:50:50.922720+0000","last_peered":"2024-04-27T07:50:50.922720+0000","last_clean":"2024-04-27T07:50:50.922720+0000","last_became_active":"2024-04-27T07:50:48.803939+0000","last_became_peered":"2024-04-27T07:50:48.803939+0000","last_unstale":"2024-04-27T07:50:50.922720+0000","last_undegraded":"2024-04-27T07:50:50.922720+0000","last_fullsized":"2024-04-27T07:50:50.922720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:38.135106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281763+0000","last_change":"2024-04-27T07:50:48.813856+0000","last_active":"2024-04-27T07:50:51.281763+0000","last_peered":"2024-04-27T07:50:51.281763+0000","last_clean":"2024-04-27T07:50:51.281763+0000","last_became_active":"2024-04-27T07:50:48.813437+0000","last_became_peered":"2024-04-27T07:50:48.813437+0000","last_unstale":"2024-04-27T07:50:51.281763+0000","last_undegraded":"2024-04-27T07:50:51.281763+0000","last_fullsized":"2024-04-27T07:50:51.281763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801741+0000","last_change":"2024-04-27T07:50:48.810807+0000","last_active":"2024-04-27T07:50:49.801741+0000","last_peered":"2024-04-27T07:50:49.801741+0000","last_clean":"2024-04-27T07:50:49.801741+0000","last_became_active":"2024-04-27T07:50:48.810545+0000","last_became_peered":"2024-04-27T07:50:48.810545+0000","last_unstale":"2024-04-27T07:50:49.801741+0000","last_undegraded":"2024-04-27T07:50:49.801741+0000","last_fullsized":"2024-04-27T07:50:49.801741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:09.348372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799832+0000","last_change":"2024-04-27T07:50:48.801260+0000","last_active":"2024-04-27T07:50:49.799832+0000","last_peered":"2024-04-27T07:50:49.799832+0000","last_clean":"2024-04-27T07:50:49.799832+0000","last_became_active":"2024-04-27T07:50:48.800565+0000","last_became_peered":"2024-04-27T07:50:48.800565+0000","last_unstale":"2024-04-27T07:50:49.799832+0000","last_undegraded":"2024-04-27T07:50:49.799832+0000","last_fullsized":"2024-04-27T07:50:49.799832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797791+0000","last_change":"2024-04-27T07:50:48.806722+0000","last_active":"2024-04-27T07:50:49.797791+0000","last_peered":"2024-04-27T07:50:49.797791+0000","last_clean":"2024-04-27T07:50:49.797791+0000","last_became_active":"2024-04-27T07:50:48.805899+0000","last_became_peered":"2024-04-27T07:50:48.805899+0000","last_unstale":"2024-04-27T07:50:49.797791+0000","last_undegraded":"2024-04-27T07:50:49.797791+0000","last_fullsized":"2024-04-27T07:50:49.797791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:28:00.765509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283852+0000","last_change":"2024-04-27T07:50:48.806262+0000","last_active":"2024-04-27T07:50:51.283852+0000","last_peered":"2024-04-27T07:50:51.283852+0000","last_clean":"2024-04-27T07:50:51.283852+0000","last_became_active":"2024-04-27T07:50:48.805987+0000","last_became_peered":"2024-04-27T07:50:48.805987+0000","last_unstale":"2024-04-27T07:50:51.283852+0000","last_undegraded":"2024-04-27T07:50:51.283852+0000","last_fullsized":"2024-04-27T07:50:51.283852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:49:40.263257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445516+0000","last_change":"2024-04-27T07:50:48.811277+0000","last_active":"2024-04-27T07:50:50.445516+0000","last_peered":"2024-04-27T07:50:50.445516+0000","last_clean":"2024-04-27T07:50:50.445516+0000","last_became_active":"2024-04-27T07:50:48.810296+0000","last_became_peered":"2024-04-27T07:50:48.810296+0000","last_unstale":"2024-04-27T07:50:50.445516+0000","last_undegraded":"2024-04-27T07:50:50.445516+0000","last_fullsized":"2024-04-27T07:50:50.445516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:43:55.556626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922970+0000","last_change":"2024-04-27T07:50:48.798022+0000","last_active":"2024-04-27T07:50:50.922970+0000","last_peered":"2024-04-27T07:50:50.922970+0000","last_clean":"2024-04-27T07:50:50.922970+0000","last_became_active":"2024-04-27T07:50:48.797464+0000","last_became_peered":"2024-04-27T07:50:48.797464+0000","last_unstale":"2024-04-27T07:50:50.922970+0000","last_undegraded":"2024-04-27T07:50:50.922970+0000","last_fullsized":"2024-04-27T07:50:50.922970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:39:13.996688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800758+0000","last_change":"2024-04-27T07:50:48.796045+0000","last_active":"2024-04-27T07:50:49.800758+0000","last_peered":"2024-04-27T07:50:49.800758+0000","last_clean":"2024-04-27T07:50:49.800758+0000","last_became_active":"2024-04-27T07:50:48.794825+0000","last_became_peered":"2024-04-27T07:50:48.794825+0000","last_unstale":"2024-04-27T07:50:49.800758+0000","last_undegraded":"2024-04-27T07:50:49.800758+0000","last_fullsized":"2024-04-27T07:50:49.800758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:49.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281332+0000","last_change":"2024-04-27T07:50:48.821433+0000","last_active":"2024-04-27T07:50:51.281332+0000","last_peered":"2024-04-27T07:50:51.281332+0000","last_clean":"2024-04-27T07:50:51.281332+0000","last_became_active":"2024-04-27T07:50:48.820660+0000","last_became_peered":"2024-04-27T07:50:48.820660+0000","last_unstale":"2024-04-27T07:50:51.281332+0000","last_undegraded":"2024-04-27T07:50:51.281332+0000","last_fullsized":"2024-04-27T07:50:51.281332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799368+0000","last_change":"2024-04-27T07:50:48.810201+0000","last_active":"2024-04-27T07:50:49.799368+0000","last_peered":"2024-04-27T07:50:49.799368+0000","last_clean":"2024-04-27T07:50:49.799368+0000","last_became_active":"2024-04-27T07:50:48.809937+0000","last_became_peered":"2024-04-27T07:50:48.809937+0000","last_unstale":"2024-04-27T07:50:49.799368+0000","last_undegraded":"2024-04-27T07:50:49.799368+0000","last_fullsized":"2024-04-27T07:50:49.799368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283175+0000","last_change":"2024-04-27T07:50:48.810567+0000","last_active":"2024-04-27T07:50:51.283175+0000","last_peered":"2024-04-27T07:50:51.283175+0000","last_clean":"2024-04-27T07:50:51.283175+0000","last_became_active":"2024-04-27T07:50:48.809921+0000","last_became_peered":"2024-04-27T07:50:48.809921+0000","last_unstale":"2024-04-27T07:50:51.283175+0000","last_undegraded":"2024-04-27T07:50:51.283175+0000","last_fullsized":"2024-04-27T07:50:51.283175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446226+0000","last_change":"2024-04-27T07:50:48.805860+0000","last_active":"2024-04-27T07:50:50.446226+0000","last_peered":"2024-04-27T07:50:50.446226+0000","last_clean":"2024-04-27T07:50:50.446226+0000","last_became_active":"2024-04-27T07:50:48.805182+0000","last_became_peered":"2024-04-27T07:50:48.805182+0000","last_unstale":"2024-04-27T07:50:50.446226+0000","last_undegraded":"2024-04-27T07:50:50.446226+0000","last_fullsized":"2024-04-27T07:50:50.446226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:10:53.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281842+0000","last_change":"2024-04-27T07:50:48.811553+0000","last_active":"2024-04-27T07:50:51.281842+0000","last_peered":"2024-04-27T07:50:51.281842+0000","last_clean":"2024-04-27T07:50:51.281842+0000","last_became_active":"2024-04-27T07:50:48.810935+0000","last_became_peered":"2024-04-27T07:50:48.810935+0000","last_unstale":"2024-04-27T07:50:51.281842+0000","last_undegraded":"2024-04-27T07:50:51.281842+0000","last_fullsized":"2024-04-27T07:50:51.281842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283139+0000","last_change":"2024-04-27T07:50:48.820547+0000","last_active":"2024-04-27T07:50:51.283139+0000","last_peered":"2024-04-27T07:50:51.283139+0000","last_clean":"2024-04-27T07:50:51.283139+0000","last_became_active":"2024-04-27T07:50:48.820170+0000","last_became_peered":"2024-04-27T07:50:48.820170+0000","last_unstale":"2024-04-27T07:50:51.283139+0000","last_undegraded":"2024-04-27T07:50:51.283139+0000","last_fullsized":"2024-04-27T07:50:51.283139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923336+0000","last_change":"2024-04-27T07:50:48.821676+0000","last_active":"2024-04-27T07:50:50.923336+0000","last_peered":"2024-04-27T07:50:50.923336+0000","last_clean":"2024-04-27T07:50:50.923336+0000","last_became_active":"2024-04-27T07:50:48.820884+0000","last_became_peered":"2024-04-27T07:50:48.820884+0000","last_unstale":"2024-04-27T07:50:50.923336+0000","last_undegraded":"2024-04-27T07:50:50.923336+0000","last_fullsized":"2024-04-27T07:50:50.923336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800884+0000","last_change":"2024-04-27T07:50:48.808706+0000","last_active":"2024-04-27T07:50:49.800884+0000","last_peered":"2024-04-27T07:50:49.800884+0000","last_clean":"2024-04-27T07:50:49.800884+0000","last_became_active":"2024-04-27T07:50:48.804475+0000","last_became_peered":"2024-04-27T07:50:48.804475+0000","last_unstale":"2024-04-27T07:50:49.800884+0000","last_undegraded":"2024-04-27T07:50:49.800884+0000","last_fullsized":"2024-04-27T07:50:49.800884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:07:50.589385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797415+0000","last_change":"2024-04-27T07:50:48.810016+0000","last_active":"2024-04-27T07:50:49.797415+0000","last_peered":"2024-04-27T07:50:49.797415+0000","last_clean":"2024-04-27T07:50:49.797415+0000","last_became_active":"2024-04-27T07:50:48.809618+0000","last_became_peered":"2024-04-27T07:50:48.809618+0000","last_unstale":"2024-04-27T07:50:49.797415+0000","last_undegraded":"2024-04-27T07:50:49.797415+0000","last_fullsized":"2024-04-27T07:50:49.797415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:20.530307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445841+0000","last_change":"2024-04-27T07:50:48.805951+0000","last_active":"2024-04-27T07:50:50.445841+0000","last_peered":"2024-04-27T07:50:50.445841+0000","last_clean":"2024-04-27T07:50:50.445841+0000","last_became_active":"2024-04-27T07:50:48.805493+0000","last_became_peered":"2024-04-27T07:50:48.805493+0000","last_unstale":"2024-04-27T07:50:50.445841+0000","last_undegraded":"2024-04-27T07:50:50.445841+0000","last_fullsized":"2024-04-27T07:50:50.445841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:42:36.766605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800350+0000","last_change":"2024-04-27T07:50:48.799052+0000","last_active":"2024-04-27T07:50:49.800350+0000","last_peered":"2024-04-27T07:50:49.800350+0000","last_clean":"2024-04-27T07:50:49.800350+0000","last_became_active":"2024-04-27T07:50:48.798769+0000","last_became_peered":"2024-04-27T07:50:48.798769+0000","last_unstale":"2024-04-27T07:50:49.800350+0000","last_undegraded":"2024-04-27T07:50:49.800350+0000","last_fullsized":"2024-04-27T07:50:49.800350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:19.179469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282661+0000","last_change":"2024-04-27T07:50:47.787481+0000","last_active":"2024-04-27T07:50:51.282661+0000","last_peered":"2024-04-27T07:50:51.282661+0000","last_clean":"2024-04-27T07:50:51.282661+0000","last_became_active":"2024-04-27T07:50:47.787133+0000","last_became_peered":"2024-04-27T07:50:47.787133+0000","last_unstale":"2024-04-27T07:50:51.282661+0000","last_undegraded":"2024-04-27T07:50:51.282661+0000","last_fullsized":"2024-04-27T07:50:51.282661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:50.164872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923143+0000","last_change":"2024-04-27T07:50:48.812526+0000","last_active":"2024-04-27T07:50:50.923143+0000","last_peered":"2024-04-27T07:50:50.923143+0000","last_clean":"2024-04-27T07:50:50.923143+0000","last_became_active":"2024-04-27T07:50:48.812099+0000","last_became_peered":"2024-04-27T07:50:48.812099+0000","last_unstale":"2024-04-27T07:50:50.923143+0000","last_undegraded":"2024-04-27T07:50:50.923143+0000","last_fullsized":"2024-04-27T07:50:50.923143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445014+0000","last_change":"2024-04-27T07:50:47.786268+0000","last_active":"2024-04-27T07:50:50.445014+0000","last_peered":"2024-04-27T07:50:50.445014+0000","last_clean":"2024-04-27T07:50:50.445014+0000","last_became_active":"2024-04-27T07:50:47.785467+0000","last_became_peered":"2024-04-27T07:50:47.785467+0000","last_unstale":"2024-04-27T07:50:50.445014+0000","last_undegraded":"2024-04-27T07:50:50.445014+0000","last_fullsized":"2024-04-27T07:50:50.445014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283343+0000","last_change":"2024-04-27T07:50:47.782977+0000","last_active":"2024-04-27T07:50:51.283343+0000","last_peered":"2024-04-27T07:50:51.283343+0000","last_clean":"2024-04-27T07:50:51.283343+0000","last_became_active":"2024-04-27T07:50:47.782680+0000","last_became_peered":"2024-04-27T07:50:47.782680+0000","last_unstale":"2024-04-27T07:50:51.283343+0000","last_undegraded":"2024-04-27T07:50:51.283343+0000","last_fullsized":"2024-04-27T07:50:51.283343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281283+0000","last_change":"2024-04-27T07:50:48.795764+0000","last_active":"2024-04-27T07:50:51.281283+0000","last_peered":"2024-04-27T07:50:51.281283+0000","last_clean":"2024-04-27T07:50:51.281283+0000","last_became_active":"2024-04-27T07:50:48.794946+0000","last_became_peered":"2024-04-27T07:50:48.794946+0000","last_unstale":"2024-04-27T07:50:51.281283+0000","last_undegraded":"2024-04-27T07:50:51.281283+0000","last_fullsized":"2024-04-27T07:50:51.281283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799501+0000","last_change":"2024-04-27T07:50:47.768634+0000","last_active":"2024-04-27T07:50:49.799501+0000","last_peered":"2024-04-27T07:50:49.799501+0000","last_clean":"2024-04-27T07:50:49.799501+0000","last_became_active":"2024-04-27T07:50:47.767344+0000","last_became_peered":"2024-04-27T07:50:47.767344+0000","last_unstale":"2024-04-27T07:50:49.799501+0000","last_undegraded":"2024-04-27T07:50:49.799501+0000","last_fullsized":"2024-04-27T07:50:49.799501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281548+0000","last_change":"2024-04-27T07:50:48.831910+0000","last_active":"2024-04-27T07:50:51.281548+0000","last_peered":"2024-04-27T07:50:51.281548+0000","last_clean":"2024-04-27T07:50:51.281548+0000","last_became_active":"2024-04-27T07:50:48.831339+0000","last_became_peered":"2024-04-27T07:50:48.831339+0000","last_unstale":"2024-04-27T07:50:51.281548+0000","last_undegraded":"2024-04-27T07:50:51.281548+0000","last_fullsized":"2024-04-27T07:50:51.281548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445402+0000","last_change":"2024-04-27T07:50:47.784442+0000","last_active":"2024-04-27T07:50:50.445402+0000","last_peered":"2024-04-27T07:50:50.445402+0000","last_clean":"2024-04-27T07:50:50.445402+0000","last_became_active":"2024-04-27T07:50:47.783769+0000","last_became_peered":"2024-04-27T07:50:47.783769+0000","last_unstale":"2024-04-27T07:50:50.445402+0000","last_undegraded":"2024-04-27T07:50:50.445402+0000","last_fullsized":"2024-04-27T07:50:50.445402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:15.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799300+0000","last_change":"2024-04-27T07:50:48.806497+0000","last_active":"2024-04-27T07:50:49.799300+0000","last_peered":"2024-04-27T07:50:49.799300+0000","last_clean":"2024-04-27T07:50:49.799300+0000","last_became_active":"2024-04-27T07:50:48.806172+0000","last_became_peered":"2024-04-27T07:50:48.806172+0000","last_unstale":"2024-04-27T07:50:49.799300+0000","last_undegraded":"2024-04-27T07:50:49.799300+0000","last_fullsized":"2024-04-27T07:50:49.799300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799348+0000","last_change":"2024-04-27T07:50:47.776964+0000","last_active":"2024-04-27T07:50:49.799348+0000","last_peered":"2024-04-27T07:50:49.799348+0000","last_clean":"2024-04-27T07:50:49.799348+0000","last_became_active":"2024-04-27T07:50:47.776179+0000","last_became_peered":"2024-04-27T07:50:47.776179+0000","last_unstale":"2024-04-27T07:50:49.799348+0000","last_undegraded":"2024-04-27T07:50:49.799348+0000","last_fullsized":"2024-04-27T07:50:49.799348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923673+0000","last_change":"2024-04-27T07:50:48.812739+0000","last_active":"2024-04-27T07:50:50.923673+0000","last_peered":"2024-04-27T07:50:50.923673+0000","last_clean":"2024-04-27T07:50:50.923673+0000","last_became_active":"2024-04-27T07:50:48.812469+0000","last_became_peered":"2024-04-27T07:50:48.812469+0000","last_unstale":"2024-04-27T07:50:50.923673+0000","last_undegraded":"2024-04-27T07:50:50.923673+0000","last_fullsized":"2024-04-27T07:50:50.923673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281980+0000","last_change":"2024-04-27T07:50:47.774587+0000","last_active":"2024-04-27T07:50:51.281980+0000","last_peered":"2024-04-27T07:50:51.281980+0000","last_clean":"2024-04-27T07:50:51.281980+0000","last_became_active":"2024-04-27T07:50:47.769083+0000","last_became_peered":"2024-04-27T07:50:47.769083+0000","last_unstale":"2024-04-27T07:50:51.281980+0000","last_undegraded":"2024-04-27T07:50:51.281980+0000","last_fullsized":"2024-04-27T07:50:51.281980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800330+0000","last_change":"2024-04-27T07:50:48.798248+0000","last_active":"2024-04-27T07:50:49.800330+0000","last_peered":"2024-04-27T07:50:49.800330+0000","last_clean":"2024-04-27T07:50:49.800330+0000","last_became_active":"2024-04-27T07:50:48.797987+0000","last_became_peered":"2024-04-27T07:50:48.797987+0000","last_unstale":"2024-04-27T07:50:49.800330+0000","last_undegraded":"2024-04-27T07:50:49.800330+0000","last_fullsized":"2024-04-27T07:50:49.800330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:28.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445367+0000","last_change":"2024-04-27T07:50:47.783980+0000","last_active":"2024-04-27T07:50:50.445367+0000","last_peered":"2024-04-27T07:50:50.445367+0000","last_clean":"2024-04-27T07:50:50.445367+0000","last_became_active":"2024-04-27T07:50:47.783127+0000","last_became_peered":"2024-04-27T07:50:47.783127+0000","last_unstale":"2024-04-27T07:50:50.445367+0000","last_undegraded":"2024-04-27T07:50:50.445367+0000","last_fullsized":"2024-04-27T07:50:50.445367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283118+0000","last_change":"2024-04-27T07:50:48.822749+0000","last_active":"2024-04-27T07:50:51.283118+0000","last_peered":"2024-04-27T07:50:51.283118+0000","last_clean":"2024-04-27T07:50:51.283118+0000","last_became_active":"2024-04-27T07:50:48.822309+0000","last_became_peered":"2024-04-27T07:50:48.822309+0000","last_unstale":"2024-04-27T07:50:51.283118+0000","last_undegraded":"2024-04-27T07:50:51.283118+0000","last_fullsized":"2024-04-27T07:50:51.283118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:43.311205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923198+0000","last_change":"2024-04-27T07:50:47.783527+0000","last_active":"2024-04-27T07:50:50.923198+0000","last_peered":"2024-04-27T07:50:50.923198+0000","last_clean":"2024-04-27T07:50:50.923198+0000","last_became_active":"2024-04-27T07:50:47.782770+0000","last_became_peered":"2024-04-27T07:50:47.782770+0000","last_unstale":"2024-04-27T07:50:50.923198+0000","last_undegraded":"2024-04-27T07:50:50.923198+0000","last_fullsized":"2024-04-27T07:50:50.923198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:30:37.164246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923078+0000","last_change":"2024-04-27T07:50:48.797984+0000","last_active":"2024-04-27T07:50:50.923078+0000","last_peered":"2024-04-27T07:50:50.923078+0000","last_clean":"2024-04-27T07:50:50.923078+0000","last_became_active":"2024-04-27T07:50:48.797194+0000","last_became_peered":"2024-04-27T07:50:48.797194+0000","last_unstale":"2024-04-27T07:50:50.923078+0000","last_undegraded":"2024-04-27T07:50:50.923078+0000","last_fullsized":"2024-04-27T07:50:50.923078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800940+0000","last_change":"2024-04-27T07:50:47.790007+0000","last_active":"2024-04-27T07:50:49.800940+0000","last_peered":"2024-04-27T07:50:49.800940+0000","last_clean":"2024-04-27T07:50:49.800940+0000","last_became_active":"2024-04-27T07:50:47.789403+0000","last_became_peered":"2024-04-27T07:50:47.789403+0000","last_unstale":"2024-04-27T07:50:49.800940+0000","last_undegraded":"2024-04-27T07:50:49.800940+0000","last_fullsized":"2024-04-27T07:50:49.800940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283254+0000","last_change":"2024-04-27T07:50:48.810819+0000","last_active":"2024-04-27T07:50:51.283254+0000","last_peered":"2024-04-27T07:50:51.283254+0000","last_clean":"2024-04-27T07:50:51.283254+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.283254+0000","last_undegraded":"2024-04-27T07:50:51.283254+0000","last_fullsized":"2024-04-27T07:50:51.283254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:50.867045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.444990+0000","last_change":"2024-04-27T07:50:47.777426+0000","last_active":"2024-04-27T07:50:50.444990+0000","last_peered":"2024-04-27T07:50:50.444990+0000","last_clean":"2024-04-27T07:50:50.444990+0000","last_became_active":"2024-04-27T07:50:47.776724+0000","last_became_peered":"2024-04-27T07:50:47.776724+0000","last_unstale":"2024-04-27T07:50:50.444990+0000","last_undegraded":"2024-04-27T07:50:50.444990+0000","last_fullsized":"2024-04-27T07:50:50.444990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:37:47.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445715+0000","last_change":"2024-04-27T07:50:48.793186+0000","last_active":"2024-04-27T07:50:50.445715+0000","last_peered":"2024-04-27T07:50:50.445715+0000","last_clean":"2024-04-27T07:50:50.445715+0000","last_became_active":"2024-04-27T07:50:48.792919+0000","last_became_peered":"2024-04-27T07:50:48.792919+0000","last_unstale":"2024-04-27T07:50:50.445715+0000","last_undegraded":"2024-04-27T07:50:50.445715+0000","last_fullsized":"2024-04-27T07:50:50.445715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:34:56.089422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797034+0000","last_change":"2024-04-27T07:50:47.770430+0000","last_active":"2024-04-27T07:50:49.797034+0000","last_peered":"2024-04-27T07:50:49.797034+0000","last_clean":"2024-04-27T07:50:49.797034+0000","last_became_active":"2024-04-27T07:50:47.770009+0000","last_became_peered":"2024-04-27T07:50:47.770009+0000","last_unstale":"2024-04-27T07:50:49.797034+0000","last_undegraded":"2024-04-27T07:50:49.797034+0000","last_fullsized":"2024-04-27T07:50:49.797034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282964+0000","last_change":"2024-04-27T07:50:48.817834+0000","last_active":"2024-04-27T07:50:51.282964+0000","last_peered":"2024-04-27T07:50:51.282964+0000","last_clean":"2024-04-27T07:50:51.282964+0000","last_became_active":"2024-04-27T07:50:48.817436+0000","last_became_peered":"2024-04-27T07:50:48.817436+0000","last_unstale":"2024-04-27T07:50:51.282964+0000","last_undegraded":"2024-04-27T07:50:51.282964+0000","last_fullsized":"2024-04-27T07:50:51.282964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:51.766684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801830+0000","last_change":"2024-04-27T07:50:47.776874+0000","last_active":"2024-04-27T07:50:49.801830+0000","last_peered":"2024-04-27T07:50:49.801830+0000","last_clean":"2024-04-27T07:50:49.801830+0000","last_became_active":"2024-04-27T07:50:47.776056+0000","last_became_peered":"2024-04-27T07:50:47.776056+0000","last_unstale":"2024-04-27T07:50:49.801830+0000","last_undegraded":"2024-04-27T07:50:49.801830+0000","last_fullsized":"2024-04-27T07:50:49.801830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:13:34.610739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283821+0000","last_change":"2024-04-27T07:50:48.812568+0000","last_active":"2024-04-27T07:50:51.283821+0000","last_peered":"2024-04-27T07:50:51.283821+0000","last_clean":"2024-04-27T07:50:51.283821+0000","last_became_active":"2024-04-27T07:50:48.811223+0000","last_became_peered":"2024-04-27T07:50:48.811223+0000","last_unstale":"2024-04-27T07:50:51.283821+0000","last_undegraded":"2024-04-27T07:50:51.283821+0000","last_fullsized":"2024-04-27T07:50:51.283821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:21:47.632417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797924+0000","last_change":"2024-04-27T07:50:47.757707+0000","last_active":"2024-04-27T07:50:49.797924+0000","last_peered":"2024-04-27T07:50:49.797924+0000","last_clean":"2024-04-27T07:50:49.797924+0000","last_became_active":"2024-04-27T07:50:47.757001+0000","last_became_peered":"2024-04-27T07:50:47.757001+0000","last_unstale":"2024-04-27T07:50:49.797924+0000","last_undegraded":"2024-04-27T07:50:49.797924+0000","last_fullsized":"2024-04-27T07:50:49.797924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283932+0000","last_change":"2024-04-27T07:50:48.820601+0000","last_active":"2024-04-27T07:50:51.283932+0000","last_peered":"2024-04-27T07:50:51.283932+0000","last_clean":"2024-04-27T07:50:51.283932+0000","last_became_active":"2024-04-27T07:50:48.817456+0000","last_became_peered":"2024-04-27T07:50:48.817456+0000","last_unstale":"2024-04-27T07:50:51.283932+0000","last_undegraded":"2024-04-27T07:50:51.283932+0000","last_fullsized":"2024-04-27T07:50:51.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:14:53.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282269+0000","last_change":"2024-04-27T07:50:47.779473+0000","last_active":"2024-04-27T07:50:51.282269+0000","last_peered":"2024-04-27T07:50:51.282269+0000","last_clean":"2024-04-27T07:50:51.282269+0000","last_became_active":"2024-04-27T07:50:47.779210+0000","last_became_peered":"2024-04-27T07:50:47.779210+0000","last_unstale":"2024-04-27T07:50:51.282269+0000","last_undegraded":"2024-04-27T07:50:51.282269+0000","last_fullsized":"2024-04-27T07:50:51.282269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801103+0000","last_change":"2024-04-27T07:50:48.822640+0000","last_active":"2024-04-27T07:50:49.801103+0000","last_peered":"2024-04-27T07:50:49.801103+0000","last_clean":"2024-04-27T07:50:49.801103+0000","last_became_active":"2024-04-27T07:50:48.822329+0000","last_became_peered":"2024-04-27T07:50:48.822329+0000","last_unstale":"2024-04-27T07:50:49.801103+0000","last_undegraded":"2024-04-27T07:50:49.801103+0000","last_fullsized":"2024-04-27T07:50:49.801103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:29.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797405+0000","last_change":"2024-04-27T07:50:47.767538+0000","last_active":"2024-04-27T07:50:49.797405+0000","last_peered":"2024-04-27T07:50:49.797405+0000","last_clean":"2024-04-27T07:50:49.797405+0000","last_became_active":"2024-04-27T07:50:47.767085+0000","last_became_peered":"2024-04-27T07:50:47.767085+0000","last_unstale":"2024-04-27T07:50:49.797405+0000","last_undegraded":"2024-04-27T07:50:49.797405+0000","last_fullsized":"2024-04-27T07:50:49.797405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283419+0000","last_change":"2024-04-27T07:50:48.818495+0000","last_active":"2024-04-27T07:50:51.283419+0000","last_peered":"2024-04-27T07:50:51.283419+0000","last_clean":"2024-04-27T07:50:51.283419+0000","last_became_active":"2024-04-27T07:50:48.817986+0000","last_became_peered":"2024-04-27T07:50:48.817986+0000","last_unstale":"2024-04-27T07:50:51.283419+0000","last_undegraded":"2024-04-27T07:50:51.283419+0000","last_fullsized":"2024-04-27T07:50:51.283419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:39:30.392854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282765+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:51.282765+0000","last_peered":"2024-04-27T07:50:51.282765+0000","last_clean":"2024-04-27T07:50:51.282765+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:51.282765+0000","last_undegraded":"2024-04-27T07:50:51.282765+0000","last_fullsized":"2024-04-27T07:50:51.282765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.798836+0000","last_change":"2024-04-27T07:50:47.765622+0000","last_active":"2024-04-27T07:50:49.798836+0000","last_peered":"2024-04-27T07:50:49.798836+0000","last_clean":"2024-04-27T07:50:49.798836+0000","last_became_active":"2024-04-27T07:50:47.764321+0000","last_became_peered":"2024-04-27T07:50:47.764321+0000","last_unstale":"2024-04-27T07:50:49.798836+0000","last_undegraded":"2024-04-27T07:50:49.798836+0000","last_fullsized":"2024-04-27T07:50:49.798836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282843+0000","last_change":"2024-04-27T07:50:48.829189+0000","last_active":"2024-04-27T07:50:51.282843+0000","last_peered":"2024-04-27T07:50:51.282843+0000","last_clean":"2024-04-27T07:50:51.282843+0000","last_became_active":"2024-04-27T07:50:48.828497+0000","last_became_peered":"2024-04-27T07:50:48.828497+0000","last_unstale":"2024-04-27T07:50:51.282843+0000","last_undegraded":"2024-04-27T07:50:51.282843+0000","last_fullsized":"2024-04-27T07:50:51.282843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282400+0000","last_change":"2024-04-27T07:50:47.801435+0000","last_active":"2024-04-27T07:50:51.282400+0000","last_peered":"2024-04-27T07:50:51.282400+0000","last_clean":"2024-04-27T07:50:51.282400+0000","last_became_active":"2024-04-27T07:50:47.800976+0000","last_became_peered":"2024-04-27T07:50:47.800976+0000","last_unstale":"2024-04-27T07:50:51.282400+0000","last_undegraded":"2024-04-27T07:50:51.282400+0000","last_fullsized":"2024-04-27T07:50:51.282400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922633+0000","last_change":"2024-04-27T07:50:48.813418+0000","last_active":"2024-04-27T07:50:50.922633+0000","last_peered":"2024-04-27T07:50:50.922633+0000","last_clean":"2024-04-27T07:50:50.922633+0000","last_became_active":"2024-04-27T07:50:48.812995+0000","last_became_peered":"2024-04-27T07:50:48.812995+0000","last_unstale":"2024-04-27T07:50:50.922633+0000","last_undegraded":"2024-04-27T07:50:50.922633+0000","last_fullsized":"2024-04-27T07:50:50.922633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:09:23.204607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797401+0000","last_change":"2024-04-27T07:50:47.753103+0000","last_active":"2024-04-27T07:50:49.797401+0000","last_peered":"2024-04-27T07:50:49.797401+0000","last_clean":"2024-04-27T07:50:49.797401+0000","last_became_active":"2024-04-27T07:50:47.752481+0000","last_became_peered":"2024-04-27T07:50:47.752481+0000","last_unstale":"2024-04-27T07:50:49.797401+0000","last_undegraded":"2024-04-27T07:50:49.797401+0000","last_fullsized":"2024-04-27T07:50:49.797401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281299+0000","last_change":"2024-04-27T07:50:48.806850+0000","last_active":"2024-04-27T07:50:51.281299+0000","last_peered":"2024-04-27T07:50:51.281299+0000","last_clean":"2024-04-27T07:50:51.281299+0000","last_became_active":"2024-04-27T07:50:48.801965+0000","last_became_peered":"2024-04-27T07:50:48.801965+0000","last_unstale":"2024-04-27T07:50:51.281299+0000","last_undegraded":"2024-04-27T07:50:51.281299+0000","last_fullsized":"2024-04-27T07:50:51.281299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801000+0000","last_change":"2024-04-27T07:50:47.779751+0000","last_active":"2024-04-27T07:50:49.801000+0000","last_peered":"2024-04-27T07:50:49.801000+0000","last_clean":"2024-04-27T07:50:49.801000+0000","last_became_active":"2024-04-27T07:50:47.778788+0000","last_became_peered":"2024-04-27T07:50:47.778788+0000","last_unstale":"2024-04-27T07:50:49.801000+0000","last_undegraded":"2024-04-27T07:50:49.801000+0000","last_fullsized":"2024-04-27T07:50:49.801000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801821+0000","last_change":"2024-04-27T07:50:48.821201+0000","last_active":"2024-04-27T07:50:49.801821+0000","last_peered":"2024-04-27T07:50:49.801821+0000","last_clean":"2024-04-27T07:50:49.801821+0000","last_became_active":"2024-04-27T07:50:48.820923+0000","last_became_peered":"2024-04-27T07:50:48.820923+0000","last_unstale":"2024-04-27T07:50:49.801821+0000","last_undegraded":"2024-04-27T07:50:49.801821+0000","last_fullsized":"2024-04-27T07:50:49.801821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:25.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281927+0000","last_change":"2024-04-27T07:50:47.769907+0000","last_active":"2024-04-27T07:50:51.281927+0000","last_peered":"2024-04-27T07:50:51.281927+0000","last_clean":"2024-04-27T07:50:51.281927+0000","last_became_active":"2024-04-27T07:50:47.769126+0000","last_became_peered":"2024-04-27T07:50:47.769126+0000","last_unstale":"2024-04-27T07:50:51.281927+0000","last_undegraded":"2024-04-27T07:50:51.281927+0000","last_fullsized":"2024-04-27T07:50:51.281927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800281+0000","last_change":"2024-04-27T07:50:48.804829+0000","last_active":"2024-04-27T07:50:49.800281+0000","last_peered":"2024-04-27T07:50:49.800281+0000","last_clean":"2024-04-27T07:50:49.800281+0000","last_became_active":"2024-04-27T07:50:48.804092+0000","last_became_peered":"2024-04-27T07:50:48.804092+0000","last_unstale":"2024-04-27T07:50:49.800281+0000","last_undegraded":"2024-04-27T07:50:49.800281+0000","last_fullsized":"2024-04-27T07:50:49.800281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283754+0000","last_change":"2024-04-27T07:50:47.801329+0000","last_active":"2024-04-27T07:50:51.283754+0000","last_peered":"2024-04-27T07:50:51.283754+0000","last_clean":"2024-04-27T07:50:51.283754+0000","last_became_active":"2024-04-27T07:50:47.800993+0000","last_became_peered":"2024-04-27T07:50:47.800993+0000","last_unstale":"2024-04-27T07:50:51.283754+0000","last_undegraded":"2024-04-27T07:50:51.283754+0000","last_fullsized":"2024-04-27T07:50:51.283754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:52:57.849530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283493+0000","last_change":"2024-04-27T07:50:48.795907+0000","last_active":"2024-04-27T07:50:51.283493+0000","last_peered":"2024-04-27T07:50:51.283493+0000","last_clean":"2024-04-27T07:50:51.283493+0000","last_became_active":"2024-04-27T07:50:48.795249+0000","last_became_peered":"2024-04-27T07:50:48.795249+0000","last_unstale":"2024-04-27T07:50:51.283493+0000","last_undegraded":"2024-04-27T07:50:51.283493+0000","last_fullsized":"2024-04-27T07:50:51.283493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282216+0000","last_change":"2024-04-27T07:50:47.774585+0000","last_active":"2024-04-27T07:50:51.282216+0000","last_peered":"2024-04-27T07:50:51.282216+0000","last_clean":"2024-04-27T07:50:51.282216+0000","last_became_active":"2024-04-27T07:50:47.774213+0000","last_became_peered":"2024-04-27T07:50:47.774213+0000","last_unstale":"2024-04-27T07:50:51.282216+0000","last_undegraded":"2024-04-27T07:50:51.282216+0000","last_fullsized":"2024-04-27T07:50:51.282216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283467+0000","last_change":"2024-04-27T07:50:48.797717+0000","last_active":"2024-04-27T07:50:51.283467+0000","last_peered":"2024-04-27T07:50:51.283467+0000","last_clean":"2024-04-27T07:50:51.283467+0000","last_became_active":"2024-04-27T07:50:48.797011+0000","last_became_peered":"2024-04-27T07:50:48.797011+0000","last_unstale":"2024-04-27T07:50:51.283467+0000","last_undegraded":"2024-04-27T07:50:51.283467+0000","last_fullsized":"2024-04-27T07:50:51.283467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445562+0000","last_change":"2024-04-27T07:50:47.796196+0000","last_active":"2024-04-27T07:50:50.445562+0000","last_peered":"2024-04-27T07:50:50.445562+0000","last_clean":"2024-04-27T07:50:50.445562+0000","last_became_active":"2024-04-27T07:50:47.795881+0000","last_became_peered":"2024-04-27T07:50:47.795881+0000","last_unstale":"2024-04-27T07:50:50.445562+0000","last_undegraded":"2024-04-27T07:50:50.445562+0000","last_fullsized":"2024-04-27T07:50:50.445562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:30:11.967127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923685+0000","last_change":"2024-04-27T07:50:48.804448+0000","last_active":"2024-04-27T07:50:50.923685+0000","last_peered":"2024-04-27T07:50:50.923685+0000","last_clean":"2024-04-27T07:50:50.923685+0000","last_became_active":"2024-04-27T07:50:48.803586+0000","last_became_peered":"2024-04-27T07:50:48.803586+0000","last_unstale":"2024-04-27T07:50:50.923685+0000","last_undegraded":"2024-04-27T07:50:50.923685+0000","last_fullsized":"2024-04-27T07:50:50.923685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282373+0000","last_change":"2024-04-27T07:50:47.796755+0000","last_active":"2024-04-27T07:50:51.282373+0000","last_peered":"2024-04-27T07:50:51.282373+0000","last_clean":"2024-04-27T07:50:51.282373+0000","last_became_active":"2024-04-27T07:50:47.796096+0000","last_became_peered":"2024-04-27T07:50:47.796096+0000","last_unstale":"2024-04-27T07:50:51.282373+0000","last_undegraded":"2024-04-27T07:50:51.282373+0000","last_fullsized":"2024-04-27T07:50:51.282373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799129+0000","last_change":"2024-04-27T07:50:48.792778+0000","last_active":"2024-04-27T07:50:49.799129+0000","last_peered":"2024-04-27T07:50:49.799129+0000","last_clean":"2024-04-27T07:50:49.799129+0000","last_became_active":"2024-04-27T07:50:48.791993+0000","last_became_peered":"2024-04-27T07:50:48.791993+0000","last_unstale":"2024-04-27T07:50:49.799129+0000","last_undegraded":"2024-04-27T07:50:49.799129+0000","last_fullsized":"2024-04-27T07:50:49.799129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:02.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281677+0000","last_change":"2024-04-27T07:50:47.785296+0000","last_active":"2024-04-27T07:50:51.281677+0000","last_peered":"2024-04-27T07:50:51.281677+0000","last_clean":"2024-04-27T07:50:51.281677+0000","last_became_active":"2024-04-27T07:50:47.784874+0000","last_became_peered":"2024-04-27T07:50:47.784874+0000","last_unstale":"2024-04-27T07:50:51.281677+0000","last_undegraded":"2024-04-27T07:50:51.281677+0000","last_fullsized":"2024-04-27T07:50:51.281677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281489+0000","last_change":"2024-04-27T07:50:48.809822+0000","last_active":"2024-04-27T07:50:51.281489+0000","last_peered":"2024-04-27T07:50:51.281489+0000","last_clean":"2024-04-27T07:50:51.281489+0000","last_became_active":"2024-04-27T07:50:48.809495+0000","last_became_peered":"2024-04-27T07:50:48.809495+0000","last_unstale":"2024-04-27T07:50:51.281489+0000","last_undegraded":"2024-04-27T07:50:51.281489+0000","last_fullsized":"2024-04-27T07:50:51.281489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281879+0000","last_change":"2024-04-27T07:50:47.777251+0000","last_active":"2024-04-27T07:50:51.281879+0000","last_peered":"2024-04-27T07:50:51.281879+0000","last_clean":"2024-04-27T07:50:51.281879+0000","last_became_active":"2024-04-27T07:50:47.776893+0000","last_became_peered":"2024-04-27T07:50:47.776893+0000","last_unstale":"2024-04-27T07:50:51.281879+0000","last_undegraded":"2024-04-27T07:50:51.281879+0000","last_fullsized":"2024-04-27T07:50:51.281879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282509+0000","last_change":"2024-04-27T07:50:48.821019+0000","last_active":"2024-04-27T07:50:51.282509+0000","last_peered":"2024-04-27T07:50:51.282509+0000","last_clean":"2024-04-27T07:50:51.282509+0000","last_became_active":"2024-04-27T07:50:48.820606+0000","last_became_peered":"2024-04-27T07:50:48.820606+0000","last_unstale":"2024-04-27T07:50:51.282509+0000","last_undegraded":"2024-04-27T07:50:51.282509+0000","last_fullsized":"2024-04-27T07:50:51.282509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283602+0000","last_change":"2024-04-27T07:50:47.800388+0000","last_active":"2024-04-27T07:50:51.283602+0000","last_peered":"2024-04-27T07:50:51.283602+0000","last_clean":"2024-04-27T07:50:51.283602+0000","last_became_active":"2024-04-27T07:50:47.799604+0000","last_became_peered":"2024-04-27T07:50:47.799604+0000","last_unstale":"2024-04-27T07:50:51.283602+0000","last_undegraded":"2024-04-27T07:50:51.283602+0000","last_fullsized":"2024-04-27T07:50:51.283602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:34:31.234332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445740+0000","last_change":"2024-04-27T07:50:48.810527+0000","last_active":"2024-04-27T07:50:50.445740+0000","last_peered":"2024-04-27T07:50:50.445740+0000","last_clean":"2024-04-27T07:50:50.445740+0000","last_became_active":"2024-04-27T07:50:48.810203+0000","last_became_peered":"2024-04-27T07:50:48.810203+0000","last_unstale":"2024-04-27T07:50:50.445740+0000","last_undegraded":"2024-04-27T07:50:50.445740+0000","last_fullsized":"2024-04-27T07:50:50.445740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:59.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799691+0000","last_change":"2024-04-27T07:50:48.787926+0000","last_active":"2024-04-27T07:50:49.799691+0000","last_peered":"2024-04-27T07:50:49.799691+0000","last_clean":"2024-04-27T07:50:49.799691+0000","last_became_active":"2024-04-27T07:50:48.787213+0000","last_became_peered":"2024-04-27T07:50:48.787213+0000","last_unstale":"2024-04-27T07:50:49.799691+0000","last_undegraded":"2024-04-27T07:50:49.799691+0000","last_fullsized":"2024-04-27T07:50:49.799691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282943+0000","last_change":"2024-04-27T07:50:47.799584+0000","last_active":"2024-04-27T07:50:51.282943+0000","last_peered":"2024-04-27T07:50:51.282943+0000","last_clean":"2024-04-27T07:50:51.282943+0000","last_became_active":"2024-04-27T07:50:47.799182+0000","last_became_peered":"2024-04-27T07:50:47.799182+0000","last_unstale":"2024-04-27T07:50:51.282943+0000","last_undegraded":"2024-04-27T07:50:51.282943+0000","last_fullsized":"2024-04-27T07:50:51.282943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:31:29.207109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801755+0000","last_change":"2024-04-27T07:50:48.798910+0000","last_active":"2024-04-27T07:50:49.801755+0000","last_peered":"2024-04-27T07:50:49.801755+0000","last_clean":"2024-04-27T07:50:49.801755+0000","last_became_active":"2024-04-27T07:50:48.798376+0000","last_became_peered":"2024-04-27T07:50:48.798376+0000","last_unstale":"2024-04-27T07:50:49.801755+0000","last_undegraded":"2024-04-27T07:50:49.801755+0000","last_fullsized":"2024-04-27T07:50:49.801755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:17:52.504141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283271+0000","last_change":"2024-04-27T07:50:47.798503+0000","last_active":"2024-04-27T07:50:51.283271+0000","last_peered":"2024-04-27T07:50:51.283271+0000","last_clean":"2024-04-27T07:50:51.283271+0000","last_became_active":"2024-04-27T07:50:47.798171+0000","last_became_peered":"2024-04-27T07:50:47.798171+0000","last_unstale":"2024-04-27T07:50:51.283271+0000","last_undegraded":"2024-04-27T07:50:51.283271+0000","last_fullsized":"2024-04-27T07:50:51.283271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:16.910153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923642+0000","last_change":"2024-04-27T07:50:48.818082+0000","last_active":"2024-04-27T07:50:50.923642+0000","last_peered":"2024-04-27T07:50:50.923642+0000","last_clean":"2024-04-27T07:50:50.923642+0000","last_became_active":"2024-04-27T07:50:48.813287+0000","last_became_peered":"2024-04-27T07:50:48.813287+0000","last_unstale":"2024-04-27T07:50:50.923642+0000","last_undegraded":"2024-04-27T07:50:50.923642+0000","last_fullsized":"2024-04-27T07:50:50.923642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282997+0000","last_change":"2024-04-27T07:50:47.770647+0000","last_active":"2024-04-27T07:50:51.282997+0000","last_peered":"2024-04-27T07:50:51.282997+0000","last_clean":"2024-04-27T07:50:51.282997+0000","last_became_active":"2024-04-27T07:50:47.770223+0000","last_became_peered":"2024-04-27T07:50:47.770223+0000","last_unstale":"2024-04-27T07:50:51.282997+0000","last_undegraded":"2024-04-27T07:50:51.282997+0000","last_fullsized":"2024-04-27T07:50:51.282997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282890+0000","last_change":"2024-04-27T07:50:48.828744+0000","last_active":"2024-04-27T07:50:51.282890+0000","last_peered":"2024-04-27T07:50:51.282890+0000","last_clean":"2024-04-27T07:50:51.282890+0000","last_became_active":"2024-04-27T07:50:48.810916+0000","last_became_peered":"2024-04-27T07:50:48.810916+0000","last_unstale":"2024-04-27T07:50:51.282890+0000","last_undegraded":"2024-04-27T07:50:51.282890+0000","last_fullsized":"2024-04-27T07:50:51.282890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923774+0000","last_change":"2024-04-27T07:50:47.785683+0000","last_active":"2024-04-27T07:50:50.923774+0000","last_peered":"2024-04-27T07:50:50.923774+0000","last_clean":"2024-04-27T07:50:50.923774+0000","last_became_active":"2024-04-27T07:50:47.784993+0000","last_became_peered":"2024-04-27T07:50:47.784993+0000","last_unstale":"2024-04-27T07:50:50.923774+0000","last_undegraded":"2024-04-27T07:50:50.923774+0000","last_fullsized":"2024-04-27T07:50:50.923774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:49.046466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:50:56.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:56.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:57.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:50:57.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 Processor -- start 2024-04-27T07:50:57.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 -- start start 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7718162da0 0x7f771806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771806e630 con 0x7f771806fc70 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771806e910 con 0x7f77180ce260 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771f60d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7718163180 con 0x7f7718162da0 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f771806fc70 0x7f77180b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36068/0 (socket says 172.21.15.22:36068) 2024-04-27T07:50:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 -- 172.21.15.22:0/2284511534 learned_addr learned my addr 172.21.15.22:0/2284511534 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771d3a9640 1 --2- 172.21.15.22:0/2284511534 >> v2:172.21.15.183:3300/0 conn(0x7f7718162da0 0x7f771806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 246796473 0 0) 0x7f771806e630 con 0x7f771806fc70 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7704003530 con 0x7f771806fc70 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2985876178 0 0) 0x7f7704003530 con 0x7f771806fc70 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 >> v2:172.21.15.183:3300/0 conn(0x7f7718162da0 msgr2=0x7f771806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 --2- 172.21.15.22:0/2284511534 >> v2:172.21.15.183:3300/0 conn(0x7f7718162da0 0x7f771806e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 >> v1:172.21.15.183:6789/0 conn(0x7f77180ce260 legacy=0x7f771806f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771806ebf0 con 0x7f771806fc70 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7710001680 con 0x7f771806fc70 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 >> v1:172.21.15.22:6789/0 conn(0x7f771806fc70 legacy=0x7f77180b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040039c0 0x7f7704003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7704004810 0x7f7704004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f770400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f770400c040 con 0x7f77040051d0 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f770400c1f0 con 0x7f7704004810 2024-04-27T07:50:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f770400c3f0 con 0x7f77040039c0 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f770400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771cba8640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7704004810 0x7f7704004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771d3a9640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040039c0 0x7f7704003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 -- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040039c0 msgr2=0x7f7704003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040039c0 0x7f7704003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 -- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7704004810 msgr2=0x7f7704004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7704004810 0x7f7704004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771dbaa640 1 -- 172.21.15.22:0/2284511534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7704003530 con 0x7f77040051d0 2024-04-27T07:50:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.480+0000 7f771cba8640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7704004810 0x7f7704004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:57.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771dbaa640 1 -- 172.21.15.22:0/2284511534 reap_dead start 2024-04-27T07:50:57.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771dbaa640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f770400b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f771000c1c0 tx=0x7f771001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20683e3db5fc1ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710007d00 con 0x7f77040051d0 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77100a9070 con 0x7f77040051d0 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f770ffff640 1 -- 172.21.15.22:0/2284511534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100a5dc0 con 0x7f77040051d0 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 msgr2=0x7f770400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f770400b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f771000c1c0 tx=0x7f771001cc60 comp rx=0 tx=0).stop 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- 172.21.15.22:0/2284511534 shutdown_connections 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 --2- 172.21.15.22:0/2284511534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f770400b9d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- 172.21.15.22:0/2284511534 >> 172.21.15.22:0/2284511534 conn(0x7f7718067f10 msgr2=0x7f771806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- 172.21.15.22:0/2284511534 shutdown_connections 2024-04-27T07:50:57.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- 172.21.15.22:0/2284511534 wait complete. 2024-04-27T07:50:57.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 Processor -- start 2024-04-27T07:50:57.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.484+0000 7f771f60d640 1 -- start start 2024-04-27T07:50:57.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77180a9ea0 0x7f77180aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7718164c70 0x7f7718165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f771806e050 con 0x7f77040051d0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f771806f1b0 con 0x7f77180a9ea0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f771806e1d0 con 0x7f7718164c70 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44906/0 (socket says 172.21.15.22:44906) 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 -- 172.21.15.22:0/3043204137 learned_addr learned my addr 172.21.15.22:0/3043204137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771cba8640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77180a9ea0 0x7f77180aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771dbaa640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7718164c70 0x7f7718165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 -- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7718164c70 msgr2=0x7f7718165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7718164c70 0x7f7718165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 -- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77180a9ea0 msgr2=0x7f77180aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77180a9ea0 0x7f77180aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 -- 172.21.15.22:0/3043204137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180aa960 con 0x7f77040051d0 2024-04-27T07:50:57.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771d3a9640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fb0 tx=0x7f77140115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771401a020 con 0x7f77040051d0 2024-04-27T07:50:57.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7714016470 con 0x7f77040051d0 2024-04-27T07:50:57.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180aac40 con 0x7f77040051d0 2024-04-27T07:50:57.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714016840 con 0x7f77040051d0 2024-04-27T07:50:57.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77180ab200 con 0x7f77040051d0 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718068c30 con 0x7f77040051d0 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f771401d020 con 0x7f77040051d0 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 0x7f76fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.488+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7714087720 con 0x7f77040051d0 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.492+0000 7f771cba8640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 0x7f76fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.492+0000 7f771cba8640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 0x7f76fc06a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c10 tx=0x7f7708009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.496+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7714024030 con 0x7f77040051d0 2024-04-27T07:50:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.652+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77180afd70 con 0x7f76fc068380 2024-04-27T07:50:57.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.672+0000 7f770dffb640 1 -- 172.21.15.22:0/3043204137 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f77180afd70 con 0x7f76fc068380 2024-04-27T07:50:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:57.679 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:50:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 msgr2=0x7f76fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 0x7f76fc06a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c10 tx=0x7f7708009040 comp rx=0 tx=0).stop 2024-04-27T07:50:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 msgr2=0x7f77181527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fb0 tx=0x7f77140115a0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 shutdown_connections 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76fc068380 0x7f76fc06a840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7718164c70 0x7f7718165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77180a9ea0 0x7f77180aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 --2- 172.21.15.22:0/3043204137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77040051d0 0x7f77181527c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 >> 172.21.15.22:0/3043204137 conn(0x7f7718067f10 msgr2=0x7f77180b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 shutdown_connections 2024-04-27T07:50:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.676+0000 7f771f60d640 1 -- 172.21.15.22:0/3043204137 wait complete. 2024-04-27T07:50:57.697 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-27T07:50:56.150373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235696,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739024,"statfs":{"total":773094113280,"available":772852760576,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002840"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283542+0000","last_change":"2024-04-27T07:50:48.807748+0000","last_active":"2024-04-27T07:50:51.283542+0000","last_peered":"2024-04-27T07:50:51.283542+0000","last_clean":"2024-04-27T07:50:51.283542+0000","last_became_active":"2024-04-27T07:50:48.806582+0000","last_became_peered":"2024-04-27T07:50:48.806582+0000","last_unstale":"2024-04-27T07:50:51.283542+0000","last_undegraded":"2024-04-27T07:50:51.283542+0000","last_fullsized":"2024-04-27T07:50:51.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.223671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282205+0000","last_change":"2024-04-27T07:50:48.809136+0000","last_active":"2024-04-27T07:50:51.282205+0000","last_peered":"2024-04-27T07:50:51.282205+0000","last_clean":"2024-04-27T07:50:51.282205+0000","last_became_active":"2024-04-27T07:50:48.808581+0000","last_became_peered":"2024-04-27T07:50:48.808581+0000","last_unstale":"2024-04-27T07:50:51.282205+0000","last_undegraded":"2024-04-27T07:50:51.282205+0000","last_fullsized":"2024-04-27T07:50:51.282205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923394+0000","last_change":"2024-04-27T07:50:48.803232+0000","last_active":"2024-04-27T07:50:50.923394+0000","last_peered":"2024-04-27T07:50:50.923394+0000","last_clean":"2024-04-27T07:50:50.923394+0000","last_became_active":"2024-04-27T07:50:48.802828+0000","last_became_peered":"2024-04-27T07:50:48.802828+0000","last_unstale":"2024-04-27T07:50:50.923394+0000","last_undegraded":"2024-04-27T07:50:50.923394+0000","last_fullsized":"2024-04-27T07:50:50.923394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800701+0000","last_change":"2024-04-27T07:50:48.822942+0000","last_active":"2024-04-27T07:50:49.800701+0000","last_peered":"2024-04-27T07:50:49.800701+0000","last_clean":"2024-04-27T07:50:49.800701+0000","last_became_active":"2024-04-27T07:50:48.822709+0000","last_became_peered":"2024-04-27T07:50:48.822709+0000","last_unstale":"2024-04-27T07:50:49.800701+0000","last_undegraded":"2024-04-27T07:50:49.800701+0000","last_fullsized":"2024-04-27T07:50:49.800701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:38.768492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283631+0000","last_change":"2024-04-27T07:50:48.805852+0000","last_active":"2024-04-27T07:50:51.283631+0000","last_peered":"2024-04-27T07:50:51.283631+0000","last_clean":"2024-04-27T07:50:51.283631+0000","last_became_active":"2024-04-27T07:50:48.805553+0000","last_became_peered":"2024-04-27T07:50:48.805553+0000","last_unstale":"2024-04-27T07:50:51.283631+0000","last_undegraded":"2024-04-27T07:50:51.283631+0000","last_fullsized":"2024-04-27T07:50:51.283631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:19:06.682886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800634+0000","last_change":"2024-04-27T07:50:48.795482+0000","last_active":"2024-04-27T07:50:49.800634+0000","last_peered":"2024-04-27T07:50:49.800634+0000","last_clean":"2024-04-27T07:50:49.800634+0000","last_became_active":"2024-04-27T07:50:48.793813+0000","last_became_peered":"2024-04-27T07:50:48.793813+0000","last_unstale":"2024-04-27T07:50:49.800634+0000","last_undegraded":"2024-04-27T07:50:49.800634+0000","last_fullsized":"2024-04-27T07:50:49.800634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:39:48.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282786+0000","last_change":"2024-04-27T07:50:48.821409+0000","last_active":"2024-04-27T07:50:51.282786+0000","last_peered":"2024-04-27T07:50:51.282786+0000","last_clean":"2024-04-27T07:50:51.282786+0000","last_became_active":"2024-04-27T07:50:48.821162+0000","last_became_peered":"2024-04-27T07:50:48.821162+0000","last_unstale":"2024-04-27T07:50:51.282786+0000","last_undegraded":"2024-04-27T07:50:51.282786+0000","last_fullsized":"2024-04-27T07:50:51.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:20:43.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281085+0000","last_change":"2024-04-27T07:50:48.830026+0000","last_active":"2024-04-27T07:50:51.281085+0000","last_peered":"2024-04-27T07:50:51.281085+0000","last_clean":"2024-04-27T07:50:51.281085+0000","last_became_active":"2024-04-27T07:50:48.821303+0000","last_became_peered":"2024-04-27T07:50:48.821303+0000","last_unstale":"2024-04-27T07:50:51.281085+0000","last_undegraded":"2024-04-27T07:50:51.281085+0000","last_fullsized":"2024-04-27T07:50:51.281085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283709+0000","last_change":"2024-04-27T07:50:48.820925+0000","last_active":"2024-04-27T07:50:51.283709+0000","last_peered":"2024-04-27T07:50:51.283709+0000","last_clean":"2024-04-27T07:50:51.283709+0000","last_became_active":"2024-04-27T07:50:48.820575+0000","last_became_peered":"2024-04-27T07:50:48.820575+0000","last_unstale":"2024-04-27T07:50:51.283709+0000","last_undegraded":"2024-04-27T07:50:51.283709+0000","last_fullsized":"2024-04-27T07:50:51.283709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:40.499571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446110+0000","last_change":"2024-04-27T07:50:48.814093+0000","last_active":"2024-04-27T07:50:50.446110+0000","last_peered":"2024-04-27T07:50:50.446110+0000","last_clean":"2024-04-27T07:50:50.446110+0000","last_became_active":"2024-04-27T07:50:48.813643+0000","last_became_peered":"2024-04-27T07:50:48.813643+0000","last_unstale":"2024-04-27T07:50:50.446110+0000","last_undegraded":"2024-04-27T07:50:50.446110+0000","last_fullsized":"2024-04-27T07:50:50.446110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:24.403447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445905+0000","last_change":"2024-04-27T07:50:48.815381+0000","last_active":"2024-04-27T07:50:50.445905+0000","last_peered":"2024-04-27T07:50:50.445905+0000","last_clean":"2024-04-27T07:50:50.445905+0000","last_became_active":"2024-04-27T07:50:48.814574+0000","last_became_peered":"2024-04-27T07:50:48.814574+0000","last_unstale":"2024-04-27T07:50:50.445905+0000","last_undegraded":"2024-04-27T07:50:50.445905+0000","last_fullsized":"2024-04-27T07:50:50.445905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:25.162778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800227+0000","last_change":"2024-04-27T07:50:48.806044+0000","last_active":"2024-04-27T07:50:49.800227+0000","last_peered":"2024-04-27T07:50:49.800227+0000","last_clean":"2024-04-27T07:50:49.800227+0000","last_became_active":"2024-04-27T07:50:48.805705+0000","last_became_peered":"2024-04-27T07:50:48.805705+0000","last_unstale":"2024-04-27T07:50:49.800227+0000","last_undegraded":"2024-04-27T07:50:49.800227+0000","last_fullsized":"2024-04-27T07:50:49.800227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:21.183002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282310+0000","last_change":"2024-04-27T07:50:48.819990+0000","last_active":"2024-04-27T07:50:51.282310+0000","last_peered":"2024-04-27T07:50:51.282310+0000","last_clean":"2024-04-27T07:50:51.282310+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.282310+0000","last_undegraded":"2024-04-27T07:50:51.282310+0000","last_fullsized":"2024-04-27T07:50:51.282310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922720+0000","last_change":"2024-04-27T07:50:48.804618+0000","last_active":"2024-04-27T07:50:50.922720+0000","last_peered":"2024-04-27T07:50:50.922720+0000","last_clean":"2024-04-27T07:50:50.922720+0000","last_became_active":"2024-04-27T07:50:48.803939+0000","last_became_peered":"2024-04-27T07:50:48.803939+0000","last_unstale":"2024-04-27T07:50:50.922720+0000","last_undegraded":"2024-04-27T07:50:50.922720+0000","last_fullsized":"2024-04-27T07:50:50.922720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:38.135106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281763+0000","last_change":"2024-04-27T07:50:48.813856+0000","last_active":"2024-04-27T07:50:51.281763+0000","last_peered":"2024-04-27T07:50:51.281763+0000","last_clean":"2024-04-27T07:50:51.281763+0000","last_became_active":"2024-04-27T07:50:48.813437+0000","last_became_peered":"2024-04-27T07:50:48.813437+0000","last_unstale":"2024-04-27T07:50:51.281763+0000","last_undegraded":"2024-04-27T07:50:51.281763+0000","last_fullsized":"2024-04-27T07:50:51.281763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801741+0000","last_change":"2024-04-27T07:50:48.810807+0000","last_active":"2024-04-27T07:50:49.801741+0000","last_peered":"2024-04-27T07:50:49.801741+0000","last_clean":"2024-04-27T07:50:49.801741+0000","last_became_active":"2024-04-27T07:50:48.810545+0000","last_became_peered":"2024-04-27T07:50:48.810545+0000","last_unstale":"2024-04-27T07:50:49.801741+0000","last_undegraded":"2024-04-27T07:50:49.801741+0000","last_fullsized":"2024-04-27T07:50:49.801741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:09.348372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799832+0000","last_change":"2024-04-27T07:50:48.801260+0000","last_active":"2024-04-27T07:50:49.799832+0000","last_peered":"2024-04-27T07:50:49.799832+0000","last_clean":"2024-04-27T07:50:49.799832+0000","last_became_active":"2024-04-27T07:50:48.800565+0000","last_became_peered":"2024-04-27T07:50:48.800565+0000","last_unstale":"2024-04-27T07:50:49.799832+0000","last_undegraded":"2024-04-27T07:50:49.799832+0000","last_fullsized":"2024-04-27T07:50:49.799832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797791+0000","last_change":"2024-04-27T07:50:48.806722+0000","last_active":"2024-04-27T07:50:49.797791+0000","last_peered":"2024-04-27T07:50:49.797791+0000","last_clean":"2024-04-27T07:50:49.797791+0000","last_became_active":"2024-04-27T07:50:48.805899+0000","last_became_peered":"2024-04-27T07:50:48.805899+0000","last_unstale":"2024-04-27T07:50:49.797791+0000","last_undegraded":"2024-04-27T07:50:49.797791+0000","last_fullsized":"2024-04-27T07:50:49.797791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:28:00.765509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283852+0000","last_change":"2024-04-27T07:50:48.806262+0000","last_active":"2024-04-27T07:50:51.283852+0000","last_peered":"2024-04-27T07:50:51.283852+0000","last_clean":"2024-04-27T07:50:51.283852+0000","last_became_active":"2024-04-27T07:50:48.805987+0000","last_became_peered":"2024-04-27T07:50:48.805987+0000","last_unstale":"2024-04-27T07:50:51.283852+0000","last_undegraded":"2024-04-27T07:50:51.283852+0000","last_fullsized":"2024-04-27T07:50:51.283852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:49:40.263257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445516+0000","last_change":"2024-04-27T07:50:48.811277+0000","last_active":"2024-04-27T07:50:50.445516+0000","last_peered":"2024-04-27T07:50:50.445516+0000","last_clean":"2024-04-27T07:50:50.445516+0000","last_became_active":"2024-04-27T07:50:48.810296+0000","last_became_peered":"2024-04-27T07:50:48.810296+0000","last_unstale":"2024-04-27T07:50:50.445516+0000","last_undegraded":"2024-04-27T07:50:50.445516+0000","last_fullsized":"2024-04-27T07:50:50.445516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:43:55.556626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922970+0000","last_change":"2024-04-27T07:50:48.798022+0000","last_active":"2024-04-27T07:50:50.922970+0000","last_peered":"2024-04-27T07:50:50.922970+0000","last_clean":"2024-04-27T07:50:50.922970+0000","last_became_active":"2024-04-27T07:50:48.797464+0000","last_became_peered":"2024-04-27T07:50:48.797464+0000","last_unstale":"2024-04-27T07:50:50.922970+0000","last_undegraded":"2024-04-27T07:50:50.922970+0000","last_fullsized":"2024-04-27T07:50:50.922970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:39:13.996688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800758+0000","last_change":"2024-04-27T07:50:48.796045+0000","last_active":"2024-04-27T07:50:49.800758+0000","last_peered":"2024-04-27T07:50:49.800758+0000","last_clean":"2024-04-27T07:50:49.800758+0000","last_became_active":"2024-04-27T07:50:48.794825+0000","last_became_peered":"2024-04-27T07:50:48.794825+0000","last_unstale":"2024-04-27T07:50:49.800758+0000","last_undegraded":"2024-04-27T07:50:49.800758+0000","last_fullsized":"2024-04-27T07:50:49.800758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:49.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281332+0000","last_change":"2024-04-27T07:50:48.821433+0000","last_active":"2024-04-27T07:50:51.281332+0000","last_peered":"2024-04-27T07:50:51.281332+0000","last_clean":"2024-04-27T07:50:51.281332+0000","last_became_active":"2024-04-27T07:50:48.820660+0000","last_became_peered":"2024-04-27T07:50:48.820660+0000","last_unstale":"2024-04-27T07:50:51.281332+0000","last_undegraded":"2024-04-27T07:50:51.281332+0000","last_fullsized":"2024-04-27T07:50:51.281332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799368+0000","last_change":"2024-04-27T07:50:48.810201+0000","last_active":"2024-04-27T07:50:49.799368+0000","last_peered":"2024-04-27T07:50:49.799368+0000","last_clean":"2024-04-27T07:50:49.799368+0000","last_became_active":"2024-04-27T07:50:48.809937+0000","last_became_peered":"2024-04-27T07:50:48.809937+0000","last_unstale":"2024-04-27T07:50:49.799368+0000","last_undegraded":"2024-04-27T07:50:49.799368+0000","last_fullsized":"2024-04-27T07:50:49.799368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283175+0000","last_change":"2024-04-27T07:50:48.810567+0000","last_active":"2024-04-27T07:50:51.283175+0000","last_peered":"2024-04-27T07:50:51.283175+0000","last_clean":"2024-04-27T07:50:51.283175+0000","last_became_active":"2024-04-27T07:50:48.809921+0000","last_became_peered":"2024-04-27T07:50:48.809921+0000","last_unstale":"2024-04-27T07:50:51.283175+0000","last_undegraded":"2024-04-27T07:50:51.283175+0000","last_fullsized":"2024-04-27T07:50:51.283175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446226+0000","last_change":"2024-04-27T07:50:48.805860+0000","last_active":"2024-04-27T07:50:50.446226+0000","last_peered":"2024-04-27T07:50:50.446226+0000","last_clean":"2024-04-27T07:50:50.446226+0000","last_became_active":"2024-04-27T07:50:48.805182+0000","last_became_peered":"2024-04-27T07:50:48.805182+0000","last_unstale":"2024-04-27T07:50:50.446226+0000","last_undegraded":"2024-04-27T07:50:50.446226+0000","last_fullsized":"2024-04-27T07:50:50.446226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:10:53.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281842+0000","last_change":"2024-04-27T07:50:48.811553+0000","last_active":"2024-04-27T07:50:51.281842+0000","last_peered":"2024-04-27T07:50:51.281842+0000","last_clean":"2024-04-27T07:50:51.281842+0000","last_became_active":"2024-04-27T07:50:48.810935+0000","last_became_peered":"2024-04-27T07:50:48.810935+0000","last_unstale":"2024-04-27T07:50:51.281842+0000","last_undegraded":"2024-04-27T07:50:51.281842+0000","last_fullsized":"2024-04-27T07:50:51.281842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283139+0000","last_change":"2024-04-27T07:50:48.820547+0000","last_active":"2024-04-27T07:50:51.283139+0000","last_peered":"2024-04-27T07:50:51.283139+0000","last_clean":"2024-04-27T07:50:51.283139+0000","last_became_active":"2024-04-27T07:50:48.820170+0000","last_became_peered":"2024-04-27T07:50:48.820170+0000","last_unstale":"2024-04-27T07:50:51.283139+0000","last_undegraded":"2024-04-27T07:50:51.283139+0000","last_fullsized":"2024-04-27T07:50:51.283139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923336+0000","last_change":"2024-04-27T07:50:48.821676+0000","last_active":"2024-04-27T07:50:50.923336+0000","last_peered":"2024-04-27T07:50:50.923336+0000","last_clean":"2024-04-27T07:50:50.923336+0000","last_became_active":"2024-04-27T07:50:48.820884+0000","last_became_peered":"2024-04-27T07:50:48.820884+0000","last_unstale":"2024-04-27T07:50:50.923336+0000","last_undegraded":"2024-04-27T07:50:50.923336+0000","last_fullsized":"2024-04-27T07:50:50.923336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800884+0000","last_change":"2024-04-27T07:50:48.808706+0000","last_active":"2024-04-27T07:50:49.800884+0000","last_peered":"2024-04-27T07:50:49.800884+0000","last_clean":"2024-04-27T07:50:49.800884+0000","last_became_active":"2024-04-27T07:50:48.804475+0000","last_became_peered":"2024-04-27T07:50:48.804475+0000","last_unstale":"2024-04-27T07:50:49.800884+0000","last_undegraded":"2024-04-27T07:50:49.800884+0000","last_fullsized":"2024-04-27T07:50:49.800884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:07:50.589385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797415+0000","last_change":"2024-04-27T07:50:48.810016+0000","last_active":"2024-04-27T07:50:49.797415+0000","last_peered":"2024-04-27T07:50:49.797415+0000","last_clean":"2024-04-27T07:50:49.797415+0000","last_became_active":"2024-04-27T07:50:48.809618+0000","last_became_peered":"2024-04-27T07:50:48.809618+0000","last_unstale":"2024-04-27T07:50:49.797415+0000","last_undegraded":"2024-04-27T07:50:49.797415+0000","last_fullsized":"2024-04-27T07:50:49.797415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:20.530307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445841+0000","last_change":"2024-04-27T07:50:48.805951+0000","last_active":"2024-04-27T07:50:50.445841+0000","last_peered":"2024-04-27T07:50:50.445841+0000","last_clean":"2024-04-27T07:50:50.445841+0000","last_became_active":"2024-04-27T07:50:48.805493+0000","last_became_peered":"2024-04-27T07:50:48.805493+0000","last_unstale":"2024-04-27T07:50:50.445841+0000","last_undegraded":"2024-04-27T07:50:50.445841+0000","last_fullsized":"2024-04-27T07:50:50.445841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:42:36.766605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800350+0000","last_change":"2024-04-27T07:50:48.799052+0000","last_active":"2024-04-27T07:50:49.800350+0000","last_peered":"2024-04-27T07:50:49.800350+0000","last_clean":"2024-04-27T07:50:49.800350+0000","last_became_active":"2024-04-27T07:50:48.798769+0000","last_became_peered":"2024-04-27T07:50:48.798769+0000","last_unstale":"2024-04-27T07:50:49.800350+0000","last_undegraded":"2024-04-27T07:50:49.800350+0000","last_fullsized":"2024-04-27T07:50:49.800350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:19.179469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282661+0000","last_change":"2024-04-27T07:50:47.787481+0000","last_active":"2024-04-27T07:50:51.282661+0000","last_peered":"2024-04-27T07:50:51.282661+0000","last_clean":"2024-04-27T07:50:51.282661+0000","last_became_active":"2024-04-27T07:50:47.787133+0000","last_became_peered":"2024-04-27T07:50:47.787133+0000","last_unstale":"2024-04-27T07:50:51.282661+0000","last_undegraded":"2024-04-27T07:50:51.282661+0000","last_fullsized":"2024-04-27T07:50:51.282661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:50.164872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923143+0000","last_change":"2024-04-27T07:50:48.812526+0000","last_active":"2024-04-27T07:50:50.923143+0000","last_peered":"2024-04-27T07:50:50.923143+0000","last_clean":"2024-04-27T07:50:50.923143+0000","last_became_active":"2024-04-27T07:50:48.812099+0000","last_became_peered":"2024-04-27T07:50:48.812099+0000","last_unstale":"2024-04-27T07:50:50.923143+0000","last_undegraded":"2024-04-27T07:50:50.923143+0000","last_fullsized":"2024-04-27T07:50:50.923143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445014+0000","last_change":"2024-04-27T07:50:47.786268+0000","last_active":"2024-04-27T07:50:50.445014+0000","last_peered":"2024-04-27T07:50:50.445014+0000","last_clean":"2024-04-27T07:50:50.445014+0000","last_became_active":"2024-04-27T07:50:47.785467+0000","last_became_peered":"2024-04-27T07:50:47.785467+0000","last_unstale":"2024-04-27T07:50:50.445014+0000","last_undegraded":"2024-04-27T07:50:50.445014+0000","last_fullsized":"2024-04-27T07:50:50.445014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283343+0000","last_change":"2024-04-27T07:50:47.782977+0000","last_active":"2024-04-27T07:50:51.283343+0000","last_peered":"2024-04-27T07:50:51.283343+0000","last_clean":"2024-04-27T07:50:51.283343+0000","last_became_active":"2024-04-27T07:50:47.782680+0000","last_became_peered":"2024-04-27T07:50:47.782680+0000","last_unstale":"2024-04-27T07:50:51.283343+0000","last_undegraded":"2024-04-27T07:50:51.283343+0000","last_fullsized":"2024-04-27T07:50:51.283343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281283+0000","last_change":"2024-04-27T07:50:48.795764+0000","last_active":"2024-04-27T07:50:51.281283+0000","last_peered":"2024-04-27T07:50:51.281283+0000","last_clean":"2024-04-27T07:50:51.281283+0000","last_became_active":"2024-04-27T07:50:48.794946+0000","last_became_peered":"2024-04-27T07:50:48.794946+0000","last_unstale":"2024-04-27T07:50:51.281283+0000","last_undegraded":"2024-04-27T07:50:51.281283+0000","last_fullsized":"2024-04-27T07:50:51.281283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799501+0000","last_change":"2024-04-27T07:50:47.768634+0000","last_active":"2024-04-27T07:50:49.799501+0000","last_peered":"2024-04-27T07:50:49.799501+0000","last_clean":"2024-04-27T07:50:49.799501+0000","last_became_active":"2024-04-27T07:50:47.767344+0000","last_became_peered":"2024-04-27T07:50:47.767344+0000","last_unstale":"2024-04-27T07:50:49.799501+0000","last_undegraded":"2024-04-27T07:50:49.799501+0000","last_fullsized":"2024-04-27T07:50:49.799501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281548+0000","last_change":"2024-04-27T07:50:48.831910+0000","last_active":"2024-04-27T07:50:51.281548+0000","last_peered":"2024-04-27T07:50:51.281548+0000","last_clean":"2024-04-27T07:50:51.281548+0000","last_became_active":"2024-04-27T07:50:48.831339+0000","last_became_peered":"2024-04-27T07:50:48.831339+0000","last_unstale":"2024-04-27T07:50:51.281548+0000","last_undegraded":"2024-04-27T07:50:51.281548+0000","last_fullsized":"2024-04-27T07:50:51.281548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445402+0000","last_change":"2024-04-27T07:50:47.784442+0000","last_active":"2024-04-27T07:50:50.445402+0000","last_peered":"2024-04-27T07:50:50.445402+0000","last_clean":"2024-04-27T07:50:50.445402+0000","last_became_active":"2024-04-27T07:50:47.783769+0000","last_became_peered":"2024-04-27T07:50:47.783769+0000","last_unstale":"2024-04-27T07:50:50.445402+0000","last_undegraded":"2024-04-27T07:50:50.445402+0000","last_fullsized":"2024-04-27T07:50:50.445402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:15.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799300+0000","last_change":"2024-04-27T07:50:48.806497+0000","last_active":"2024-04-27T07:50:49.799300+0000","last_peered":"2024-04-27T07:50:49.799300+0000","last_clean":"2024-04-27T07:50:49.799300+0000","last_became_active":"2024-04-27T07:50:48.806172+0000","last_became_peered":"2024-04-27T07:50:48.806172+0000","last_unstale":"2024-04-27T07:50:49.799300+0000","last_undegraded":"2024-04-27T07:50:49.799300+0000","last_fullsized":"2024-04-27T07:50:49.799300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799348+0000","last_change":"2024-04-27T07:50:47.776964+0000","last_active":"2024-04-27T07:50:49.799348+0000","last_peered":"2024-04-27T07:50:49.799348+0000","last_clean":"2024-04-27T07:50:49.799348+0000","last_became_active":"2024-04-27T07:50:47.776179+0000","last_became_peered":"2024-04-27T07:50:47.776179+0000","last_unstale":"2024-04-27T07:50:49.799348+0000","last_undegraded":"2024-04-27T07:50:49.799348+0000","last_fullsized":"2024-04-27T07:50:49.799348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923673+0000","last_change":"2024-04-27T07:50:48.812739+0000","last_active":"2024-04-27T07:50:50.923673+0000","last_peered":"2024-04-27T07:50:50.923673+0000","last_clean":"2024-04-27T07:50:50.923673+0000","last_became_active":"2024-04-27T07:50:48.812469+0000","last_became_peered":"2024-04-27T07:50:48.812469+0000","last_unstale":"2024-04-27T07:50:50.923673+0000","last_undegraded":"2024-04-27T07:50:50.923673+0000","last_fullsized":"2024-04-27T07:50:50.923673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281980+0000","last_change":"2024-04-27T07:50:47.774587+0000","last_active":"2024-04-27T07:50:51.281980+0000","last_peered":"2024-04-27T07:50:51.281980+0000","last_clean":"2024-04-27T07:50:51.281980+0000","last_became_active":"2024-04-27T07:50:47.769083+0000","last_became_peered":"2024-04-27T07:50:47.769083+0000","last_unstale":"2024-04-27T07:50:51.281980+0000","last_undegraded":"2024-04-27T07:50:51.281980+0000","last_fullsized":"2024-04-27T07:50:51.281980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800330+0000","last_change":"2024-04-27T07:50:48.798248+0000","last_active":"2024-04-27T07:50:49.800330+0000","last_peered":"2024-04-27T07:50:49.800330+0000","last_clean":"2024-04-27T07:50:49.800330+0000","last_became_active":"2024-04-27T07:50:48.797987+0000","last_became_peered":"2024-04-27T07:50:48.797987+0000","last_unstale":"2024-04-27T07:50:49.800330+0000","last_undegraded":"2024-04-27T07:50:49.800330+0000","last_fullsized":"2024-04-27T07:50:49.800330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:28.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445367+0000","last_change":"2024-04-27T07:50:47.783980+0000","last_active":"2024-04-27T07:50:50.445367+0000","last_peered":"2024-04-27T07:50:50.445367+0000","last_clean":"2024-04-27T07:50:50.445367+0000","last_became_active":"2024-04-27T07:50:47.783127+0000","last_became_peered":"2024-04-27T07:50:47.783127+0000","last_unstale":"2024-04-27T07:50:50.445367+0000","last_undegraded":"2024-04-27T07:50:50.445367+0000","last_fullsized":"2024-04-27T07:50:50.445367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283118+0000","last_change":"2024-04-27T07:50:48.822749+0000","last_active":"2024-04-27T07:50:51.283118+0000","last_peered":"2024-04-27T07:50:51.283118+0000","last_clean":"2024-04-27T07:50:51.283118+0000","last_became_active":"2024-04-27T07:50:48.822309+0000","last_became_peered":"2024-04-27T07:50:48.822309+0000","last_unstale":"2024-04-27T07:50:51.283118+0000","last_undegraded":"2024-04-27T07:50:51.283118+0000","last_fullsized":"2024-04-27T07:50:51.283118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:43.311205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923198+0000","last_change":"2024-04-27T07:50:47.783527+0000","last_active":"2024-04-27T07:50:50.923198+0000","last_peered":"2024-04-27T07:50:50.923198+0000","last_clean":"2024-04-27T07:50:50.923198+0000","last_became_active":"2024-04-27T07:50:47.782770+0000","last_became_peered":"2024-04-27T07:50:47.782770+0000","last_unstale":"2024-04-27T07:50:50.923198+0000","last_undegraded":"2024-04-27T07:50:50.923198+0000","last_fullsized":"2024-04-27T07:50:50.923198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:30:37.164246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923078+0000","last_change":"2024-04-27T07:50:48.797984+0000","last_active":"2024-04-27T07:50:50.923078+0000","last_peered":"2024-04-27T07:50:50.923078+0000","last_clean":"2024-04-27T07:50:50.923078+0000","last_became_active":"2024-04-27T07:50:48.797194+0000","last_became_peered":"2024-04-27T07:50:48.797194+0000","last_unstale":"2024-04-27T07:50:50.923078+0000","last_undegraded":"2024-04-27T07:50:50.923078+0000","last_fullsized":"2024-04-27T07:50:50.923078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800940+0000","last_change":"2024-04-27T07:50:47.790007+0000","last_active":"2024-04-27T07:50:49.800940+0000","last_peered":"2024-04-27T07:50:49.800940+0000","last_clean":"2024-04-27T07:50:49.800940+0000","last_became_active":"2024-04-27T07:50:47.789403+0000","last_became_peered":"2024-04-27T07:50:47.789403+0000","last_unstale":"2024-04-27T07:50:49.800940+0000","last_undegraded":"2024-04-27T07:50:49.800940+0000","last_fullsized":"2024-04-27T07:50:49.800940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283254+0000","last_change":"2024-04-27T07:50:48.810819+0000","last_active":"2024-04-27T07:50:51.283254+0000","last_peered":"2024-04-27T07:50:51.283254+0000","last_clean":"2024-04-27T07:50:51.283254+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.283254+0000","last_undegraded":"2024-04-27T07:50:51.283254+0000","last_fullsized":"2024-04-27T07:50:51.283254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:50.867045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.444990+0000","last_change":"2024-04-27T07:50:47.777426+0000","last_active":"2024-04-27T07:50:50.444990+0000","last_peered":"2024-04-27T07:50:50.444990+0000","last_clean":"2024-04-27T07:50:50.444990+0000","last_became_active":"2024-04-27T07:50:47.776724+0000","last_became_peered":"2024-04-27T07:50:47.776724+0000","last_unstale":"2024-04-27T07:50:50.444990+0000","last_undegraded":"2024-04-27T07:50:50.444990+0000","last_fullsized":"2024-04-27T07:50:50.444990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:37:47.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445715+0000","last_change":"2024-04-27T07:50:48.793186+0000","last_active":"2024-04-27T07:50:50.445715+0000","last_peered":"2024-04-27T07:50:50.445715+0000","last_clean":"2024-04-27T07:50:50.445715+0000","last_became_active":"2024-04-27T07:50:48.792919+0000","last_became_peered":"2024-04-27T07:50:48.792919+0000","last_unstale":"2024-04-27T07:50:50.445715+0000","last_undegraded":"2024-04-27T07:50:50.445715+0000","last_fullsized":"2024-04-27T07:50:50.445715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:34:56.089422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797034+0000","last_change":"2024-04-27T07:50:47.770430+0000","last_active":"2024-04-27T07:50:49.797034+0000","last_peered":"2024-04-27T07:50:49.797034+0000","last_clean":"2024-04-27T07:50:49.797034+0000","last_became_active":"2024-04-27T07:50:47.770009+0000","last_became_peered":"2024-04-27T07:50:47.770009+0000","last_unstale":"2024-04-27T07:50:49.797034+0000","last_undegraded":"2024-04-27T07:50:49.797034+0000","last_fullsized":"2024-04-27T07:50:49.797034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282964+0000","last_change":"2024-04-27T07:50:48.817834+0000","last_active":"2024-04-27T07:50:51.282964+0000","last_peered":"2024-04-27T07:50:51.282964+0000","last_clean":"2024-04-27T07:50:51.282964+0000","last_became_active":"2024-04-27T07:50:48.817436+0000","last_became_peered":"2024-04-27T07:50:48.817436+0000","last_unstale":"2024-04-27T07:50:51.282964+0000","last_undegraded":"2024-04-27T07:50:51.282964+0000","last_fullsized":"2024-04-27T07:50:51.282964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:51.766684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801830+0000","last_change":"2024-04-27T07:50:47.776874+0000","last_active":"2024-04-27T07:50:49.801830+0000","last_peered":"2024-04-27T07:50:49.801830+0000","last_clean":"2024-04-27T07:50:49.801830+0000","last_became_active":"2024-04-27T07:50:47.776056+0000","last_became_peered":"2024-04-27T07:50:47.776056+0000","last_unstale":"2024-04-27T07:50:49.801830+0000","last_undegraded":"2024-04-27T07:50:49.801830+0000","last_fullsized":"2024-04-27T07:50:49.801830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:13:34.610739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283821+0000","last_change":"2024-04-27T07:50:48.812568+0000","last_active":"2024-04-27T07:50:51.283821+0000","last_peered":"2024-04-27T07:50:51.283821+0000","last_clean":"2024-04-27T07:50:51.283821+0000","last_became_active":"2024-04-27T07:50:48.811223+0000","last_became_peered":"2024-04-27T07:50:48.811223+0000","last_unstale":"2024-04-27T07:50:51.283821+0000","last_undegraded":"2024-04-27T07:50:51.283821+0000","last_fullsized":"2024-04-27T07:50:51.283821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:21:47.632417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797924+0000","last_change":"2024-04-27T07:50:47.757707+0000","last_active":"2024-04-27T07:50:49.797924+0000","last_peered":"2024-04-27T07:50:49.797924+0000","last_clean":"2024-04-27T07:50:49.797924+0000","last_became_active":"2024-04-27T07:50:47.757001+0000","last_became_peered":"2024-04-27T07:50:47.757001+0000","last_unstale":"2024-04-27T07:50:49.797924+0000","last_undegraded":"2024-04-27T07:50:49.797924+0000","last_fullsized":"2024-04-27T07:50:49.797924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283932+0000","last_change":"2024-04-27T07:50:48.820601+0000","last_active":"2024-04-27T07:50:51.283932+0000","last_peered":"2024-04-27T07:50:51.283932+0000","last_clean":"2024-04-27T07:50:51.283932+0000","last_became_active":"2024-04-27T07:50:48.817456+0000","last_became_peered":"2024-04-27T07:50:48.817456+0000","last_unstale":"2024-04-27T07:50:51.283932+0000","last_undegraded":"2024-04-27T07:50:51.283932+0000","last_fullsized":"2024-04-27T07:50:51.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:14:53.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282269+0000","last_change":"2024-04-27T07:50:47.779473+0000","last_active":"2024-04-27T07:50:51.282269+0000","last_peered":"2024-04-27T07:50:51.282269+0000","last_clean":"2024-04-27T07:50:51.282269+0000","last_became_active":"2024-04-27T07:50:47.779210+0000","last_became_peered":"2024-04-27T07:50:47.779210+0000","last_unstale":"2024-04-27T07:50:51.282269+0000","last_undegraded":"2024-04-27T07:50:51.282269+0000","last_fullsized":"2024-04-27T07:50:51.282269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801103+0000","last_change":"2024-04-27T07:50:48.822640+0000","last_active":"2024-04-27T07:50:49.801103+0000","last_peered":"2024-04-27T07:50:49.801103+0000","last_clean":"2024-04-27T07:50:49.801103+0000","last_became_active":"2024-04-27T07:50:48.822329+0000","last_became_peered":"2024-04-27T07:50:48.822329+0000","last_unstale":"2024-04-27T07:50:49.801103+0000","last_undegraded":"2024-04-27T07:50:49.801103+0000","last_fullsized":"2024-04-27T07:50:49.801103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:29.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797405+0000","last_change":"2024-04-27T07:50:47.767538+0000","last_active":"2024-04-27T07:50:49.797405+0000","last_peered":"2024-04-27T07:50:49.797405+0000","last_clean":"2024-04-27T07:50:49.797405+0000","last_became_active":"2024-04-27T07:50:47.767085+0000","last_became_peered":"2024-04-27T07:50:47.767085+0000","last_unstale":"2024-04-27T07:50:49.797405+0000","last_undegraded":"2024-04-27T07:50:49.797405+0000","last_fullsized":"2024-04-27T07:50:49.797405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283419+0000","last_change":"2024-04-27T07:50:48.818495+0000","last_active":"2024-04-27T07:50:51.283419+0000","last_peered":"2024-04-27T07:50:51.283419+0000","last_clean":"2024-04-27T07:50:51.283419+0000","last_became_active":"2024-04-27T07:50:48.817986+0000","last_became_peered":"2024-04-27T07:50:48.817986+0000","last_unstale":"2024-04-27T07:50:51.283419+0000","last_undegraded":"2024-04-27T07:50:51.283419+0000","last_fullsized":"2024-04-27T07:50:51.283419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:39:30.392854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282765+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:51.282765+0000","last_peered":"2024-04-27T07:50:51.282765+0000","last_clean":"2024-04-27T07:50:51.282765+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:51.282765+0000","last_undegraded":"2024-04-27T07:50:51.282765+0000","last_fullsized":"2024-04-27T07:50:51.282765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.798836+0000","last_change":"2024-04-27T07:50:47.765622+0000","last_active":"2024-04-27T07:50:49.798836+0000","last_peered":"2024-04-27T07:50:49.798836+0000","last_clean":"2024-04-27T07:50:49.798836+0000","last_became_active":"2024-04-27T07:50:47.764321+0000","last_became_peered":"2024-04-27T07:50:47.764321+0000","last_unstale":"2024-04-27T07:50:49.798836+0000","last_undegraded":"2024-04-27T07:50:49.798836+0000","last_fullsized":"2024-04-27T07:50:49.798836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282843+0000","last_change":"2024-04-27T07:50:48.829189+0000","last_active":"2024-04-27T07:50:51.282843+0000","last_peered":"2024-04-27T07:50:51.282843+0000","last_clean":"2024-04-27T07:50:51.282843+0000","last_became_active":"2024-04-27T07:50:48.828497+0000","last_became_peered":"2024-04-27T07:50:48.828497+0000","last_unstale":"2024-04-27T07:50:51.282843+0000","last_undegraded":"2024-04-27T07:50:51.282843+0000","last_fullsized":"2024-04-27T07:50:51.282843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282400+0000","last_change":"2024-04-27T07:50:47.801435+0000","last_active":"2024-04-27T07:50:51.282400+0000","last_peered":"2024-04-27T07:50:51.282400+0000","last_clean":"2024-04-27T07:50:51.282400+0000","last_became_active":"2024-04-27T07:50:47.800976+0000","last_became_peered":"2024-04-27T07:50:47.800976+0000","last_unstale":"2024-04-27T07:50:51.282400+0000","last_undegraded":"2024-04-27T07:50:51.282400+0000","last_fullsized":"2024-04-27T07:50:51.282400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922633+0000","last_change":"2024-04-27T07:50:48.813418+0000","last_active":"2024-04-27T07:50:50.922633+0000","last_peered":"2024-04-27T07:50:50.922633+0000","last_clean":"2024-04-27T07:50:50.922633+0000","last_became_active":"2024-04-27T07:50:48.812995+0000","last_became_peered":"2024-04-27T07:50:48.812995+0000","last_unstale":"2024-04-27T07:50:50.922633+0000","last_undegraded":"2024-04-27T07:50:50.922633+0000","last_fullsized":"2024-04-27T07:50:50.922633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:09:23.204607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797401+0000","last_change":"2024-04-27T07:50:47.753103+0000","last_active":"2024-04-27T07:50:49.797401+0000","last_peered":"2024-04-27T07:50:49.797401+0000","last_clean":"2024-04-27T07:50:49.797401+0000","last_became_active":"2024-04-27T07:50:47.752481+0000","last_became_peered":"2024-04-27T07:50:47.752481+0000","last_unstale":"2024-04-27T07:50:49.797401+0000","last_undegraded":"2024-04-27T07:50:49.797401+0000","last_fullsized":"2024-04-27T07:50:49.797401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281299+0000","last_change":"2024-04-27T07:50:48.806850+0000","last_active":"2024-04-27T07:50:51.281299+0000","last_peered":"2024-04-27T07:50:51.281299+0000","last_clean":"2024-04-27T07:50:51.281299+0000","last_became_active":"2024-04-27T07:50:48.801965+0000","last_became_peered":"2024-04-27T07:50:48.801965+0000","last_unstale":"2024-04-27T07:50:51.281299+0000","last_undegraded":"2024-04-27T07:50:51.281299+0000","last_fullsized":"2024-04-27T07:50:51.281299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801000+0000","last_change":"2024-04-27T07:50:47.779751+0000","last_active":"2024-04-27T07:50:49.801000+0000","last_peered":"2024-04-27T07:50:49.801000+0000","last_clean":"2024-04-27T07:50:49.801000+0000","last_became_active":"2024-04-27T07:50:47.778788+0000","last_became_peered":"2024-04-27T07:50:47.778788+0000","last_unstale":"2024-04-27T07:50:49.801000+0000","last_undegraded":"2024-04-27T07:50:49.801000+0000","last_fullsized":"2024-04-27T07:50:49.801000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801821+0000","last_change":"2024-04-27T07:50:48.821201+0000","last_active":"2024-04-27T07:50:49.801821+0000","last_peered":"2024-04-27T07:50:49.801821+0000","last_clean":"2024-04-27T07:50:49.801821+0000","last_became_active":"2024-04-27T07:50:48.820923+0000","last_became_peered":"2024-04-27T07:50:48.820923+0000","last_unstale":"2024-04-27T07:50:49.801821+0000","last_undegraded":"2024-04-27T07:50:49.801821+0000","last_fullsized":"2024-04-27T07:50:49.801821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:25.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281927+0000","last_change":"2024-04-27T07:50:47.769907+0000","last_active":"2024-04-27T07:50:51.281927+0000","last_peered":"2024-04-27T07:50:51.281927+0000","last_clean":"2024-04-27T07:50:51.281927+0000","last_became_active":"2024-04-27T07:50:47.769126+0000","last_became_peered":"2024-04-27T07:50:47.769126+0000","last_unstale":"2024-04-27T07:50:51.281927+0000","last_undegraded":"2024-04-27T07:50:51.281927+0000","last_fullsized":"2024-04-27T07:50:51.281927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800281+0000","last_change":"2024-04-27T07:50:48.804829+0000","last_active":"2024-04-27T07:50:49.800281+0000","last_peered":"2024-04-27T07:50:49.800281+0000","last_clean":"2024-04-27T07:50:49.800281+0000","last_became_active":"2024-04-27T07:50:48.804092+0000","last_became_peered":"2024-04-27T07:50:48.804092+0000","last_unstale":"2024-04-27T07:50:49.800281+0000","last_undegraded":"2024-04-27T07:50:49.800281+0000","last_fullsized":"2024-04-27T07:50:49.800281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283754+0000","last_change":"2024-04-27T07:50:47.801329+0000","last_active":"2024-04-27T07:50:51.283754+0000","last_peered":"2024-04-27T07:50:51.283754+0000","last_clean":"2024-04-27T07:50:51.283754+0000","last_became_active":"2024-04-27T07:50:47.800993+0000","last_became_peered":"2024-04-27T07:50:47.800993+0000","last_unstale":"2024-04-27T07:50:51.283754+0000","last_undegraded":"2024-04-27T07:50:51.283754+0000","last_fullsized":"2024-04-27T07:50:51.283754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:52:57.849530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283493+0000","last_change":"2024-04-27T07:50:48.795907+0000","last_active":"2024-04-27T07:50:51.283493+0000","last_peered":"2024-04-27T07:50:51.283493+0000","last_clean":"2024-04-27T07:50:51.283493+0000","last_became_active":"2024-04-27T07:50:48.795249+0000","last_became_peered":"2024-04-27T07:50:48.795249+0000","last_unstale":"2024-04-27T07:50:51.283493+0000","last_undegraded":"2024-04-27T07:50:51.283493+0000","last_fullsized":"2024-04-27T07:50:51.283493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282216+0000","last_change":"2024-04-27T07:50:47.774585+0000","last_active":"2024-04-27T07:50:51.282216+0000","last_peered":"2024-04-27T07:50:51.282216+0000","last_clean":"2024-04-27T07:50:51.282216+0000","last_became_active":"2024-04-27T07:50:47.774213+0000","last_became_peered":"2024-04-27T07:50:47.774213+0000","last_unstale":"2024-04-27T07:50:51.282216+0000","last_undegraded":"2024-04-27T07:50:51.282216+0000","last_fullsized":"2024-04-27T07:50:51.282216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283467+0000","last_change":"2024-04-27T07:50:48.797717+0000","last_active":"2024-04-27T07:50:51.283467+0000","last_peered":"2024-04-27T07:50:51.283467+0000","last_clean":"2024-04-27T07:50:51.283467+0000","last_became_active":"2024-04-27T07:50:48.797011+0000","last_became_peered":"2024-04-27T07:50:48.797011+0000","last_unstale":"2024-04-27T07:50:51.283467+0000","last_undegraded":"2024-04-27T07:50:51.283467+0000","last_fullsized":"2024-04-27T07:50:51.283467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445562+0000","last_change":"2024-04-27T07:50:47.796196+0000","last_active":"2024-04-27T07:50:50.445562+0000","last_peered":"2024-04-27T07:50:50.445562+0000","last_clean":"2024-04-27T07:50:50.445562+0000","last_became_active":"2024-04-27T07:50:47.795881+0000","last_became_peered":"2024-04-27T07:50:47.795881+0000","last_unstale":"2024-04-27T07:50:50.445562+0000","last_undegraded":"2024-04-27T07:50:50.445562+0000","last_fullsized":"2024-04-27T07:50:50.445562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:30:11.967127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923685+0000","last_change":"2024-04-27T07:50:48.804448+0000","last_active":"2024-04-27T07:50:50.923685+0000","last_peered":"2024-04-27T07:50:50.923685+0000","last_clean":"2024-04-27T07:50:50.923685+0000","last_became_active":"2024-04-27T07:50:48.803586+0000","last_became_peered":"2024-04-27T07:50:48.803586+0000","last_unstale":"2024-04-27T07:50:50.923685+0000","last_undegraded":"2024-04-27T07:50:50.923685+0000","last_fullsized":"2024-04-27T07:50:50.923685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282373+0000","last_change":"2024-04-27T07:50:47.796755+0000","last_active":"2024-04-27T07:50:51.282373+0000","last_peered":"2024-04-27T07:50:51.282373+0000","last_clean":"2024-04-27T07:50:51.282373+0000","last_became_active":"2024-04-27T07:50:47.796096+0000","last_became_peered":"2024-04-27T07:50:47.796096+0000","last_unstale":"2024-04-27T07:50:51.282373+0000","last_undegraded":"2024-04-27T07:50:51.282373+0000","last_fullsized":"2024-04-27T07:50:51.282373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799129+0000","last_change":"2024-04-27T07:50:48.792778+0000","last_active":"2024-04-27T07:50:49.799129+0000","last_peered":"2024-04-27T07:50:49.799129+0000","last_clean":"2024-04-27T07:50:49.799129+0000","last_became_active":"2024-04-27T07:50:48.791993+0000","last_became_peered":"2024-04-27T07:50:48.791993+0000","last_unstale":"2024-04-27T07:50:49.799129+0000","last_undegraded":"2024-04-27T07:50:49.799129+0000","last_fullsized":"2024-04-27T07:50:49.799129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:02.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281677+0000","last_change":"2024-04-27T07:50:47.785296+0000","last_active":"2024-04-27T07:50:51.281677+0000","last_peered":"2024-04-27T07:50:51.281677+0000","last_clean":"2024-04-27T07:50:51.281677+0000","last_became_active":"2024-04-27T07:50:47.784874+0000","last_became_peered":"2024-04-27T07:50:47.784874+0000","last_unstale":"2024-04-27T07:50:51.281677+0000","last_undegraded":"2024-04-27T07:50:51.281677+0000","last_fullsized":"2024-04-27T07:50:51.281677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281489+0000","last_change":"2024-04-27T07:50:48.809822+0000","last_active":"2024-04-27T07:50:51.281489+0000","last_peered":"2024-04-27T07:50:51.281489+0000","last_clean":"2024-04-27T07:50:51.281489+0000","last_became_active":"2024-04-27T07:50:48.809495+0000","last_became_peered":"2024-04-27T07:50:48.809495+0000","last_unstale":"2024-04-27T07:50:51.281489+0000","last_undegraded":"2024-04-27T07:50:51.281489+0000","last_fullsized":"2024-04-27T07:50:51.281489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281879+0000","last_change":"2024-04-27T07:50:47.777251+0000","last_active":"2024-04-27T07:50:51.281879+0000","last_peered":"2024-04-27T07:50:51.281879+0000","last_clean":"2024-04-27T07:50:51.281879+0000","last_became_active":"2024-04-27T07:50:47.776893+0000","last_became_peered":"2024-04-27T07:50:47.776893+0000","last_unstale":"2024-04-27T07:50:51.281879+0000","last_undegraded":"2024-04-27T07:50:51.281879+0000","last_fullsized":"2024-04-27T07:50:51.281879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282509+0000","last_change":"2024-04-27T07:50:48.821019+0000","last_active":"2024-04-27T07:50:51.282509+0000","last_peered":"2024-04-27T07:50:51.282509+0000","last_clean":"2024-04-27T07:50:51.282509+0000","last_became_active":"2024-04-27T07:50:48.820606+0000","last_became_peered":"2024-04-27T07:50:48.820606+0000","last_unstale":"2024-04-27T07:50:51.282509+0000","last_undegraded":"2024-04-27T07:50:51.282509+0000","last_fullsized":"2024-04-27T07:50:51.282509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283602+0000","last_change":"2024-04-27T07:50:47.800388+0000","last_active":"2024-04-27T07:50:51.283602+0000","last_peered":"2024-04-27T07:50:51.283602+0000","last_clean":"2024-04-27T07:50:51.283602+0000","last_became_active":"2024-04-27T07:50:47.799604+0000","last_became_peered":"2024-04-27T07:50:47.799604+0000","last_unstale":"2024-04-27T07:50:51.283602+0000","last_undegraded":"2024-04-27T07:50:51.283602+0000","last_fullsized":"2024-04-27T07:50:51.283602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:34:31.234332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445740+0000","last_change":"2024-04-27T07:50:48.810527+0000","last_active":"2024-04-27T07:50:50.445740+0000","last_peered":"2024-04-27T07:50:50.445740+0000","last_clean":"2024-04-27T07:50:50.445740+0000","last_became_active":"2024-04-27T07:50:48.810203+0000","last_became_peered":"2024-04-27T07:50:48.810203+0000","last_unstale":"2024-04-27T07:50:50.445740+0000","last_undegraded":"2024-04-27T07:50:50.445740+0000","last_fullsized":"2024-04-27T07:50:50.445740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:59.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799691+0000","last_change":"2024-04-27T07:50:48.787926+0000","last_active":"2024-04-27T07:50:49.799691+0000","last_peered":"2024-04-27T07:50:49.799691+0000","last_clean":"2024-04-27T07:50:49.799691+0000","last_became_active":"2024-04-27T07:50:48.787213+0000","last_became_peered":"2024-04-27T07:50:48.787213+0000","last_unstale":"2024-04-27T07:50:49.799691+0000","last_undegraded":"2024-04-27T07:50:49.799691+0000","last_fullsized":"2024-04-27T07:50:49.799691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282943+0000","last_change":"2024-04-27T07:50:47.799584+0000","last_active":"2024-04-27T07:50:51.282943+0000","last_peered":"2024-04-27T07:50:51.282943+0000","last_clean":"2024-04-27T07:50:51.282943+0000","last_became_active":"2024-04-27T07:50:47.799182+0000","last_became_peered":"2024-04-27T07:50:47.799182+0000","last_unstale":"2024-04-27T07:50:51.282943+0000","last_undegraded":"2024-04-27T07:50:51.282943+0000","last_fullsized":"2024-04-27T07:50:51.282943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:31:29.207109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801755+0000","last_change":"2024-04-27T07:50:48.798910+0000","last_active":"2024-04-27T07:50:49.801755+0000","last_peered":"2024-04-27T07:50:49.801755+0000","last_clean":"2024-04-27T07:50:49.801755+0000","last_became_active":"2024-04-27T07:50:48.798376+0000","last_became_peered":"2024-04-27T07:50:48.798376+0000","last_unstale":"2024-04-27T07:50:49.801755+0000","last_undegraded":"2024-04-27T07:50:49.801755+0000","last_fullsized":"2024-04-27T07:50:49.801755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:17:52.504141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283271+0000","last_change":"2024-04-27T07:50:47.798503+0000","last_active":"2024-04-27T07:50:51.283271+0000","last_peered":"2024-04-27T07:50:51.283271+0000","last_clean":"2024-04-27T07:50:51.283271+0000","last_became_active":"2024-04-27T07:50:47.798171+0000","last_became_peered":"2024-04-27T07:50:47.798171+0000","last_unstale":"2024-04-27T07:50:51.283271+0000","last_undegraded":"2024-04-27T07:50:51.283271+0000","last_fullsized":"2024-04-27T07:50:51.283271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:16.910153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923642+0000","last_change":"2024-04-27T07:50:48.818082+0000","last_active":"2024-04-27T07:50:50.923642+0000","last_peered":"2024-04-27T07:50:50.923642+0000","last_clean":"2024-04-27T07:50:50.923642+0000","last_became_active":"2024-04-27T07:50:48.813287+0000","last_became_peered":"2024-04-27T07:50:48.813287+0000","last_unstale":"2024-04-27T07:50:50.923642+0000","last_undegraded":"2024-04-27T07:50:50.923642+0000","last_fullsized":"2024-04-27T07:50:50.923642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282997+0000","last_change":"2024-04-27T07:50:47.770647+0000","last_active":"2024-04-27T07:50:51.282997+0000","last_peered":"2024-04-27T07:50:51.282997+0000","last_clean":"2024-04-27T07:50:51.282997+0000","last_became_active":"2024-04-27T07:50:47.770223+0000","last_became_peered":"2024-04-27T07:50:47.770223+0000","last_unstale":"2024-04-27T07:50:51.282997+0000","last_undegraded":"2024-04-27T07:50:51.282997+0000","last_fullsized":"2024-04-27T07:50:51.282997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282890+0000","last_change":"2024-04-27T07:50:48.828744+0000","last_active":"2024-04-27T07:50:51.282890+0000","last_peered":"2024-04-27T07:50:51.282890+0000","last_clean":"2024-04-27T07:50:51.282890+0000","last_became_active":"2024-04-27T07:50:48.810916+0000","last_became_peered":"2024-04-27T07:50:48.810916+0000","last_unstale":"2024-04-27T07:50:51.282890+0000","last_undegraded":"2024-04-27T07:50:51.282890+0000","last_fullsized":"2024-04-27T07:50:51.282890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923774+0000","last_change":"2024-04-27T07:50:47.785683+0000","last_active":"2024-04-27T07:50:50.923774+0000","last_peered":"2024-04-27T07:50:50.923774+0000","last_clean":"2024-04-27T07:50:50.923774+0000","last_became_active":"2024-04-27T07:50:47.784993+0000","last_became_peered":"2024-04-27T07:50:47.784993+0000","last_unstale":"2024-04-27T07:50:50.923774+0000","last_undegraded":"2024-04-27T07:50:50.923774+0000","last_fullsized":"2024-04-27T07:50:50.923774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:49.046466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:50:57.701 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-27T07:50:57.701 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T07:50:57.701 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-27T07:50:57.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-27T07:50:57.702 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-27T07:50:57.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 Processor -- start 2024-04-27T07:50:57.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 -- start start 2024-04-27T07:50:57.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 0x7ff2dc15ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2dc0b3e00 0x7ff2dc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2dc06d120 0x7ff2dc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc06e940 con 0x7ff2dc06d120 2024-04-27T07:50:57.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc15f650 con 0x7ff2dc0b3e00 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2e1b74640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc15f930 con 0x7ff2dc15eba0 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2dc06d120 0x7ff2dc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2dc06d120 0x7ff2dc06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44912/0 (socket says 172.21.15.22:44912) 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 -- 172.21.15.22:0/1517552316 learned_addr learned my addr 172.21.15.22:0/1517552316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2daffd640 1 --2- 172.21.15.22:0/1517552316 >> v2:172.21.15.183:3300/0 conn(0x7ff2dc0b3e00 0x7ff2dc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 0x7ff2dc15ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 -- 172.21.15.22:0/1517552316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 msgr2=0x7ff2dc15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 0x7ff2dc15ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 -- 172.21.15.22:0/1517552316 >> v2:172.21.15.183:3300/0 conn(0x7ff2dc0b3e00 msgr2=0x7ff2dc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/1517552316 >> v2:172.21.15.183:3300/0 conn(0x7ff2dc0b3e00 0x7ff2dc06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 -- 172.21.15.22:0/1517552316 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc15fc10 con 0x7ff2dc06d120 2024-04-27T07:50:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 0x7ff2dc15ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:57.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.864+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc06d120 0x7ff2dc06edf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09f130 tx=0x7ff2d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0989e5ba529dea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2da7fc640 1 -- 172.21.15.22:0/1517552316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40ab5f0 con 0x7ff2dc06d120 2024-04-27T07:50:57.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2da7fc640 1 -- 172.21.15.22:0/1517552316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40b0070 con 0x7ff2dc06d120 2024-04-27T07:50:57.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2da7fc640 1 -- 172.21.15.22:0/1517552316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40ab9b0 con 0x7ff2dc06d120 2024-04-27T07:50:57.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/1517552316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc06d120 msgr2=0x7ff2dc06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc06d120 0x7ff2dc06edf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09f130 tx=0x7ff2d401c520 comp rx=0 tx=0).stop 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/1517552316 shutdown_connections 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc06d120 0x7ff2dc06edf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/1517552316 >> v2:172.21.15.183:3300/0 conn(0x7ff2dc0b3e00 0x7ff2dc06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/1517552316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc15eba0 0x7ff2dc15ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/1517552316 >> 172.21.15.22:0/1517552316 conn(0x7ff2dc072870 msgr2=0x7ff2dc072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/1517552316 shutdown_connections 2024-04-27T07:50:57.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/1517552316 wait complete. 2024-04-27T07:50:57.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 Processor -- start 2024-04-27T07:50:57.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- start start 2024-04-27T07:50:57.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2dc06d120 0x7ff2dc0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 0x7ff2dc0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc06fed0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc06fd50 con 0x7ff2dc06d120 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc070050 con 0x7ff2dc0b3e00 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44920/0 (socket says 172.21.15.22:44920) 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 -- 172.21.15.22:0/2779678331 learned_addr learned my addr 172.21.15.22:0/2779678331 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2dc06d120 0x7ff2dc0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2daffd640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 0x7ff2dc0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 -- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 msgr2=0x7ff2dc0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 0x7ff2dc0b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 -- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2dc06d120 msgr2=0x7ff2dc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:57.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2dc06d120 0x7ff2dc0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 -- 172.21.15.22:0/2779678331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc0ced90 con 0x7ff2dc15eba0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2daffd640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 0x7ff2dc0b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2dbfff640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40ad040 tx=0x7ff2d401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40be5f0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40be7b0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc0cf020 con 0x7ff2dc15eba0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40beba0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.868+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc0cf8c0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc0cf410 con 0x7ff2dc15eba0 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2d40b43f0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 0x7ff2c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2d4120cc0 con 0x7ff2dc15eba0 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 0x7ff2c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:57.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.872+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 0x7ff2c806a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0075b0 tx=0x7ff2cc007540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:57.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:57.876+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2d40c4060 con 0x7ff2dc15eba0 2024-04-27T07:50:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.084+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff2dc069340 con 0x7ff2dc15eba0 2024-04-27T07:50:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2c3fff640 1 -- 172.21.15.22:0/2779678331 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168088 (secure 0 0 0) 0x7ff2d4016110 con 0x7ff2dc15eba0 2024-04-27T07:50:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 msgr2=0x7ff2c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 0x7ff2c806a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0075b0 tx=0x7ff2cc007540 comp rx=0 tx=0).stop 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 msgr2=0x7ff2dc0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40ad040 tx=0x7ff2d401ad60 comp rx=0 tx=0).stop 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 shutdown_connections 2024-04-27T07:50:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2c8068380 0x7ff2c806a840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2dc15eba0 0x7ff2dc0b0670 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2dc0b3e00 0x7ff2dc0b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 --2- 172.21.15.22:0/2779678331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2dc06d120 0x7ff2dc0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 >> 172.21.15.22:0/2779678331 conn(0x7ff2dc072870 msgr2=0x7ff2dc161b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.092+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 shutdown_connections 2024-04-27T07:50:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.096+0000 7ff2e1b74640 1 -- 172.21.15.22:0/2779678331 wait complete. 2024-04-27T07:50:58.116 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":832018128},{"type":"v1","addr":"172.21.15.183:6833","nonce":832018128}]},"active_addr":"172.21.15.183:6833/832018128","active_change":"2024-04-27T07:50:36.135081+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":397026720}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":1684156128}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2944025219}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":3715027542}]}]} 2024-04-27T07:50:58.119 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-27T07:50:58.119 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-27T07:50:58.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:58.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.232+0000 7fe76abbc640 1 Processor -- start 2024-04-27T07:50:58.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.232+0000 7fe76abbc640 1 -- start start 2024-04-27T07:50:58.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe76415ebe0 0x7fe7640cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 0x7fe7640d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7640b3d30 con 0x7fe7640d4790 2024-04-27T07:50:58.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe7640d4540 con 0x7fe76415ebe0 2024-04-27T07:50:58.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe7640ba7a0 con 0x7fe7640b7930 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe76415ebe0 0x7fe7640cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe768958640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe7640d4790 0x7fe7640d4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36070/0 (socket says 172.21.15.22:36070) 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe768958640 1 -- 172.21.15.22:0/695369779 learned_addr learned my addr 172.21.15.22:0/695369779 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe769159640 1 --2- 172.21.15.22:0/695369779 >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 0x7fe7640d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 -- 172.21.15.22:0/695369779 >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 msgr2=0x7fe7640d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 --2- 172.21.15.22:0/695369779 >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 0x7fe7640d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 -- 172.21.15.22:0/695369779 >> v1:172.21.15.22:6789/0 conn(0x7fe7640d4790 legacy=0x7fe7640d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe7637fe640 1 -- 172.21.15.22:0/695369779 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837759393 0 0) 0x7fe74c001120 con 0x7fe7640d4790 2024-04-27T07:50:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 -- 172.21.15.22:0/695369779 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640ba4c0 con 0x7fe76415ebe0 2024-04-27T07:50:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe769159640 1 --2- 172.21.15.22:0/695369779 >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 0x7fe7640d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe763fff640 1 --2- 172.21.15.22:0/695369779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76415ebe0 0x7fe7640cf860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b39b0 tx=0x7fe75401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a2c3fbad58bb27 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe7637fe640 1 -- 172.21.15.22:0/695369779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b1070 con 0x7fe76415ebe0 2024-04-27T07:50:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe7637fe640 1 -- 172.21.15.22:0/695369779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7540ac620 con 0x7fe76415ebe0 2024-04-27T07:50:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe7637fe640 1 -- 172.21.15.22:0/695369779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540ac9f0 con 0x7fe76415ebe0 2024-04-27T07:50:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- 172.21.15.22:0/695369779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76415ebe0 msgr2=0x7fe7640cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 --2- 172.21.15.22:0/695369779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76415ebe0 0x7fe7640cf860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b39b0 tx=0x7fe75401c3c0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- 172.21.15.22:0/695369779 shutdown_connections 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 --2- 172.21.15.22:0/695369779 >> v2:172.21.15.183:3300/0 conn(0x7fe7640b7930 0x7fe7640d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 --2- 172.21.15.22:0/695369779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76415ebe0 0x7fe7640cf860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- 172.21.15.22:0/695369779 >> 172.21.15.22:0/695369779 conn(0x7fe764067d10 msgr2=0x7fe764068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- 172.21.15.22:0/695369779 shutdown_connections 2024-04-27T07:50:58.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.236+0000 7fe76abbc640 1 -- 172.21.15.22:0/695369779 wait complete. 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 Processor -- start 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- start start 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 0x7fe764152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7640b5020 0x7fe7640cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640cff20 con 0x7fe7640d4790 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640cfda0 con 0x7fe7640b7930 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe7640d3500 con 0x7fe7640b5020 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44936/0 (socket says 172.21.15.22:44936) 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 -- 172.21.15.22:0/1324453669 learned_addr learned my addr 172.21.15.22:0/1324453669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe768958640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 0x7fe764152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe769159640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7640b5020 0x7fe7640cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 -- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7640b5020 msgr2=0x7fe7640cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7640b5020 0x7fe7640cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 -- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 msgr2=0x7fe764152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 0x7fe764152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 -- 172.21.15.22:0/1324453669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640d2a90 con 0x7fe7640d4790 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe768958640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 0x7fe764152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe763fff640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe754019ff0 tx=0x7fe754019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b1040 con 0x7fe7640d4790 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7540ba070 con 0x7fe7640d4790 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe764152d60 con 0x7fe7640d4790 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b5870 con 0x7fe7640d4790 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640b4260 con 0x7fe7640d4790 2024-04-27T07:50:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.240+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764068c30 con 0x7fe7640d4790 2024-04-27T07:50:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.244+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe7540ac4e0 con 0x7fe7640d4790 2024-04-27T07:50:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.244+0000 7fe7617fa640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 0x7fe74406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.244+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe7541209d0 con 0x7fe7640d4790 2024-04-27T07:50:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.244+0000 7fe768958640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 0x7fe74406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.244+0000 7fe768958640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 0x7fe74406a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe74c007fd0 tx=0x7fe74c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.248+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe754016070 con 0x7fe7640d4790 2024-04-27T07:50:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.412+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7640cf2c0 con 0x7fe7640d4790 2024-04-27T07:50:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe7617fa640 1 -- 172.21.15.22:0/1324453669 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fe7540f0ef0 con 0x7fe7640d4790 2024-04-27T07:50:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 msgr2=0x7fe74406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 0x7fe74406a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe74c007fd0 tx=0x7fe74c009040 comp rx=0 tx=0).stop 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 msgr2=0x7fe7640cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe754019ff0 tx=0x7fe754019160 comp rx=0 tx=0).stop 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 shutdown_connections 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe744068380 0x7fe74406a840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7640b5020 0x7fe7640cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7640d4790 0x7fe7640cd8e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 --2- 172.21.15.22:0/1324453669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7640b7930 0x7fe764152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 >> 172.21.15.22:0/1324453669 conn(0x7fe764067d10 msgr2=0x7fe76415d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 shutdown_connections 2024-04-27T07:50:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.416+0000 7fe76abbc640 1 -- 172.21.15.22:0/1324453669 wait complete. 2024-04-27T07:50:58.439 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-27T07:50:58.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:50:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 Processor -- start 2024-04-27T07:50:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 -- start start 2024-04-27T07:50:58.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f90a00d4790 0x7f90a00d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a015ebe0 0x7f90a00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a00b3d30 con 0x7f90a00b7930 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f90a00d4540 con 0x7f90a00d4790 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a6f12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00ba7a0 con 0x7f90a015ebe0 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f90a00b7930 0x7f90a00b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36080/0 (socket says 172.21.15.22:36080) 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 -- 172.21.15.22:0/1620623394 learned_addr learned my addr 172.21.15.22:0/1620623394 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a4cae640 1 --2- 172.21.15.22:0/1620623394 >> v2:172.21.15.183:3300/0 conn(0x7f90a00d4790 0x7f90a00d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909ffff640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a015ebe0 0x7f90a00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485591055 0 0) 0x7f90980014a0 con 0x7f90a00b7930 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9088003530 con 0x7f90a00b7930 2024-04-27T07:50:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3661756863 0 0) 0x7f9088003530 con 0x7f90a00b7930 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a015ebe0 msgr2=0x7f90a00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a015ebe0 0x7f90a00cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 >> v2:172.21.15.183:3300/0 conn(0x7f90a00d4790 msgr2=0x7f90a00d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 --2- 172.21.15.22:0/1620623394 >> v2:172.21.15.183:3300/0 conn(0x7f90a00d4790 0x7f90a00d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a00d50b0 con 0x7f90a00b7930 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f9098001bd0 con 0x7f90a00b7930 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 >> v1:172.21.15.22:6789/0 conn(0x7f90a00b7930 legacy=0x7f90a00b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90880039c0 0x7f9088003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088004320 0x7f9088004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9088004e60 0x7f90880052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00d4540 con 0x7f9088004e60 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00ba7a0 con 0x7f9088004320 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f908800bdf0 con 0x7f90880039c0 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9088004e60 0x7f90880052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a4cae640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90880039c0 0x7f9088003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909ffff640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088004320 0x7f9088004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 -- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90880039c0 msgr2=0x7f9088003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90880039c0 0x7f9088003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 -- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088004320 msgr2=0x7f9088004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088004320 0x7f9088004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f90a54af640 1 -- 172.21.15.22:0/1620623394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9088003530 con 0x7f9088004e60 2024-04-27T07:50:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.564+0000 7f909ffff640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088004320 0x7f9088004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9088004e60 0x7f90880052e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f909800aa60 tx=0x7f909801fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34293c4dadc7cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800a6b0 con 0x7f9088004e60 2024-04-27T07:50:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90980ad070 con 0x7f9088004e60 2024-04-27T07:50:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f909f7fe640 1 -- 172.21.15.22:0/1620623394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980a9960 con 0x7f9088004e60 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- 172.21.15.22:0/1620623394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9088004e60 msgr2=0x7f90880052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 --2- 172.21.15.22:0/1620623394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9088004e60 0x7f90880052e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f909800aa60 tx=0x7f909801fb00 comp rx=0 tx=0).stop 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 -- 172.21.15.22:0/1620623394 reap_dead start 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- 172.21.15.22:0/1620623394 shutdown_connections 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- 172.21.15.22:0/1620623394 >> 172.21.15.22:0/1620623394 conn(0x7f90a0067d10 msgr2=0x7f90a0068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- 172.21.15.22:0/1620623394 shutdown_connections 2024-04-27T07:50:58.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- 172.21.15.22:0/1620623394 wait complete. 2024-04-27T07:50:58.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 Processor -- start 2024-04-27T07:50:58.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- start start 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 0x7f90a0160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a01594d0 0x7f90a015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00d4260 con 0x7f90a015ce00 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00d3f80 con 0x7f90a01590f0 2024-04-27T07:50:58.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a6f12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00ba120 con 0x7f90a01594d0 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44958/0 (socket says 172.21.15.22:44958) 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 -- 172.21.15.22:0/52770032 learned_addr learned my addr 172.21.15.22:0/52770032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a4cae640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 0x7f90a0160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 -- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a01594d0 msgr2=0x7f90a015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f909ffff640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a01594d0 0x7f90a015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a01594d0 0x7f90a015c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 -- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 msgr2=0x7f90a0160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 0x7f90a0160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a54af640 1 -- 172.21.15.22:0/52770032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a015ef50 con 0x7f90a015ce00 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.568+0000 7f90a4cae640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 0x7f90a0160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a54af640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f909800aa60 tx=0x7f9088005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800a6b0 con 0x7f90a015ce00 2024-04-27T07:50:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90980a9720 con 0x7f90a015ce00 2024-04-27T07:50:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a015f230 con 0x7f90a015ce00 2024-04-27T07:50:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980011d0 con 0x7f90a015ce00 2024-04-27T07:50:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a015f7f0 con 0x7f90a015ce00 2024-04-27T07:50:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a0068c30 con 0x7f90a015ce00 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f90980a9ad0 con 0x7f90a015ce00 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 0x7f908c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9098123800 con 0x7f90a015ce00 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a4cae640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 0x7f908c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.572+0000 7f90a4cae640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 0x7f908c06a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9090009aa0 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.576+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f90a00b3d30 con 0x7f90a015ce00 2024-04-27T07:50:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.736+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90a006d6e0 con 0x7f90a015ce00 2024-04-27T07:50:58.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.740+0000 7f909d7fa640 1 -- 172.21.15.22:0/52770032 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f90980f3d30 con 0x7f90a015ce00 2024-04-27T07:50:58.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:50:58.746 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:50:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 msgr2=0x7f908c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 0x7f908c06a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9090009aa0 tx=0x7f9090008040 comp rx=0 tx=0).stop 2024-04-27T07:50:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 msgr2=0x7f90a01f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f909800aa60 tx=0x7f9088005840 comp rx=0 tx=0).stop 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 shutdown_connections 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f908c0683b0 0x7f908c06a870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90a015ce00 0x7f90a01f7570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90a01594d0 0x7f90a015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 --2- 172.21.15.22:0/52770032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90a01590f0 0x7f90a0160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 >> 172.21.15.22:0/52770032 conn(0x7f90a0067d10 msgr2=0x7f90a00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 shutdown_connections 2024-04-27T07:50:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:58.744+0000 7f90a6f12640 1 -- 172.21.15.22:0/52770032 wait complete. 2024-04-27T07:50:58.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-27T07:50:58.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-27T07:50:58.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-27T07:50:58.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-27T07:50:58.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-27T07:50:58.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-27T07:50:58.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-27T07:50:58.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-27T07:50:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 Processor -- start 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 -- start start 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb98400aae0 0x7fb98404ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 0x7fb9840e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb9840df690 con 0x7fb98400aae0 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb984054120 con 0x7fb984033d00 2024-04-27T07:50:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98bfff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb984054400 con 0x7fb9840df2b0 2024-04-27T07:50:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7f79c2f5c640 1 Processor -- start 2024-04-27T07:50:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7f79c2f5c640 1 -- start start 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb98400aae0 0x7fb98404ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:52596/0 (socket says 172.21.15.22:52596) 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98affd640 1 -- 172.21.15.22:0/4026528839 learned_addr learned my addr 172.21.15.22:0/4026528839 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98b7fe640 1 --2- 172.21.15.22:0/4026528839 >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 0x7fb9840e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 -- 172.21.15.22:0/4026528839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 msgr2=0x7fb98404faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 -- 172.21.15.22:0/4026528839 >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 msgr2=0x7fb9840e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/4026528839 >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 0x7fb9840e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.156+0000 7fb98a7fc640 1 -- 172.21.15.22:0/4026528839 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9840546e0 con 0x7fb98400aae0 2024-04-27T07:50:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7fb98b7fe640 1 --2- 172.21.15.22:0/4026528839 >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 0x7fb9840e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7fb98affd640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c2f5c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79b404bba0 0x7f79b4006980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c2f5c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b4050e10 con 0x7f79b40339c0 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c2f5c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40510f0 con 0x7f79b40e1020 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c2f5c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f79b4005ea0 con 0x7f79b404bba0 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c275b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f79b40339c0 0x7f79b4006ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36096/0 (socket says 172.21.15.22:36096) 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c275b640 1 -- 172.21.15.22:0/563357279 learned_addr learned my addr 172.21.15.22:0/563357279 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c1759640 1 --2- 172.21.15.22:0/563357279 >> v2:172.21.15.22:3300/0 conn(0x7f79b404bba0 0x7f79b4006980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c0f58640 1 -- 172.21.15.22:0/563357279 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282107273 0 0) 0x7f79b4050e10 con 0x7f79b40339c0 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c1759640 1 -- 172.21.15.22:0/563357279 >> v1:172.21.15.183:6789/0 conn(0x7f79b40e1020 legacy=0x7f79b4039df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c1759640 1 -- 172.21.15.22:0/563357279 >> v1:172.21.15.22:6789/0 conn(0x7f79b40339c0 legacy=0x7f79b4006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f79c1759640 1 -- 172.21.15.22:0/563357279 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b4006180 con 0x7f79b404bba0 2024-04-27T07:50:59.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 Processor -- start 2024-04-27T07:50:59.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 -- start start 2024-04-27T07:50:59.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f124015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1240070f30 0x7f124015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f124015ff20 0x7f124006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f124015ff20 0x7f124006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f124015ff20 0x7f124006f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44988/0 (socket says 172.21.15.22:44988) 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 -- 172.21.15.22:0/1516932875 learned_addr learned my addr 172.21.15.22:0/1516932875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1240070490 con 0x7f124015ff20 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f12400cffc0 con 0x7f1240070f30 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f12400b3f00 con 0x7f124015ecf0 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246160640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f124015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb98400aae0 0x7fb98404ffe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9800097c0 tx=0x7fb98001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97bc6d6d48aff22 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f124595f640 1 --2- 172.21.15.22:0/1516932875 >> v2:172.21.15.183:3300/0 conn(0x7f1240070f30 0x7f124015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 -- 172.21.15.22:0/1516932875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 msgr2=0x7f124015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f124015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 -- 172.21.15.22:0/1516932875 >> v2:172.21.15.183:3300/0 conn(0x7f1240070f30 msgr2=0x7f124015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 --2- 172.21.15.22:0/1516932875 >> v2:172.21.15.183:3300/0 conn(0x7f1240070f30 0x7f124015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.160+0000 7f1246961640 1 -- 172.21.15.22:0/1516932875 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12400706e0 con 0x7f124015ff20 2024-04-27T07:50:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb989ffb640 1 -- 172.21.15.22:0/4026528839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800b0070 con 0x7fb98400aae0 2024-04-27T07:50:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c1759640 1 --2- 172.21.15.22:0/563357279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b404bba0 0x7f79b4006980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79b80097c0 tx=0x7f79b801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dbea4f1a7aef91 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c0f58640 1 -- 172.21.15.22:0/563357279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b80b0070 con 0x7f79b404bba0 2024-04-27T07:50:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c0f58640 1 -- 172.21.15.22:0/563357279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b80ab600 con 0x7f79b404bba0 2024-04-27T07:50:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c0f58640 1 -- 172.21.15.22:0/563357279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b80ab9d0 con 0x7f79b404bba0 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb989ffb640 1 -- 172.21.15.22:0/4026528839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9800ab600 con 0x7fb98400aae0 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb989ffb640 1 -- 172.21.15.22:0/4026528839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800ab9d0 con 0x7fb98400aae0 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- 172.21.15.22:0/4026528839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb98400aae0 msgr2=0x7fb98404ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb98400aae0 0x7fb98404ffe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9800097c0 tx=0x7fb98001c340 comp rx=0 tx=0).stop 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- 172.21.15.22:0/4026528839 shutdown_connections 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 --2- 172.21.15.22:0/4026528839 >> v2:172.21.15.183:3300/0 conn(0x7fb984033d00 0x7fb9840e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb98400aae0 0x7fb98404ffe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- 172.21.15.22:0/563357279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b404bba0 msgr2=0x7f79b4006980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 --2- 172.21.15.22:0/563357279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b404bba0 0x7f79b4006980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79b80097c0 tx=0x7f79b801c4c0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 --2- 172.21.15.22:0/4026528839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9840df2b0 0x7fb98404faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- 172.21.15.22:0/4026528839 >> 172.21.15.22:0/4026528839 conn(0x7fb984009fc0 msgr2=0x7fb98400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- 172.21.15.22:0/4026528839 shutdown_connections 2024-04-27T07:50:59.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 Processor -- start 2024-04-27T07:50:59.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- 172.21.15.22:0/563357279 shutdown_connections 2024-04-27T07:50:59.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 --2- 172.21.15.22:0/563357279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b404bba0 0x7f79b4006980 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- 172.21.15.22:0/563357279 >> 172.21.15.22:0/563357279 conn(0x7f79b4009fc0 msgr2=0x7f79b400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- 172.21.15.22:0/563357279 shutdown_connections 2024-04-27T07:50:59.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f1246961640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f124006f160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f124006dda0 tx=0x7f123801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd096559b397cd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- 172.21.15.22:0/4026528839 wait complete. 2024-04-27T07:50:59.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 Processor -- start 2024-04-27T07:50:59.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f124515e640 1 -- 172.21.15.22:0/1516932875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12380ac070 con 0x7f124015ff20 2024-04-27T07:50:59.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- 172.21.15.22:0/563357279 wait complete. 2024-04-27T07:50:59.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 Processor -- start 2024-04-27T07:50:59.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 -- start start 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70380d0d80 0x7f70380b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70380ad1a0 0x7f703815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703815aca0 con 0x7f703815a8c0 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f703815f3b0 con 0x7f70380d0d80 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f703d57a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f70380afc60 con 0x7f70380ad1a0 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70380d0d80 0x7f70380b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70380d0d80 0x7f70380b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44992/0 (socket says 172.21.15.22:44992) 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 -- 172.21.15.22:0/754490002 learned_addr learned my addr 172.21.15.22:0/754490002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f124515e640 1 -- 172.21.15.22:0/1516932875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12380a8420 con 0x7f124015ff20 2024-04-27T07:50:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f124515e640 1 -- 172.21.15.22:0/1516932875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12380a87f0 con 0x7f124015ff20 2024-04-27T07:50:59.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7fb98bfff640 1 -- start start 2024-04-27T07:50:59.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 msgr2=0x7f124006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f124006f160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f124006dda0 tx=0x7f123801c730 comp rx=0 tx=0).stop 2024-04-27T07:50:59.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb98400aae0 0x7fb9841885f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 0x7fb9840e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9840df2b0 0x7fb9840e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb98404f660 con 0x7fb984033d00 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb984053dd0 con 0x7fb9840df2b0 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb98404f7e0 con 0x7fb98400aae0 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 0x7fb9840e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 0x7fb9840e18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45000/0 (socket says 172.21.15.22:45000) 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 -- 172.21.15.22:0/3121092839 learned_addr learned my addr 172.21.15.22:0/3121092839 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 -- 172.21.15.22:0/754490002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70380ad1a0 msgr2=0x7f703815ee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 --2- 172.21.15.22:0/754490002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70380ad1a0 0x7f703815ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 -- 172.21.15.22:0/754490002 >> v1:172.21.15.183:6789/0 conn(0x7f703815a8c0 legacy=0x7f70380d0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f70377fe640 1 -- 172.21.15.22:0/754490002 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70380b1100 con 0x7f70380d0d80 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.164+0000 7f79c2f5c640 1 -- start start 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b40339c0 0x7f79b404a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79b404bba0 0x7f79b4036750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b40e1020 0x7f79b4036c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40ef2b0 con 0x7f79b40e1020 2024-04-27T07:50:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79b403a750 con 0x7f79b40339c0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f79b40ef430 con 0x7f79b404bba0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c1f5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b40339c0 0x7f79b404a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c1f5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b40339c0 0x7f79b404a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54626/0 (socket says 172.21.15.22:54626) 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c1f5a640 1 -- 172.21.15.22:0/3814103389 learned_addr learned my addr 172.21.15.22:0/3814103389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b40e1020 0x7f79b4036c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98affd640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb98400aae0 0x7fb9841885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98b7fe640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9840df2b0 0x7fb9840e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 -- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb98400aae0 msgr2=0x7fb9841885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb98400aae0 0x7fb9841885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 -- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9840df2b0 msgr2=0x7fb9840e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9840df2b0 0x7fb9840e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb984189880 con 0x7fb984033d00 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 shutdown_connections 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f124006f160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 --2- 172.21.15.22:0/1516932875 >> v2:172.21.15.183:3300/0 conn(0x7f1240070f30 0x7f124015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 --2- 172.21.15.22:0/1516932875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f124015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 >> 172.21.15.22:0/1516932875 conn(0x7f12400b3020 msgr2=0x7f12400b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98a7fc640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 0x7fb9840e18c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb98001a9f0 tx=0x7fb9800190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 -- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79b404bba0 msgr2=0x7f79b4036750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79b404bba0 0x7f79b4036750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 -- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b40339c0 msgr2=0x7f79b404a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b40339c0 0x7f79b404a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b404a560 con 0x7f79b40e1020 2024-04-27T07:50:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 shutdown_connections 2024-04-27T07:50:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800b8020 con 0x7fb984033d00 2024-04-27T07:50:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9800b4600 con 0x7fb984033d00 2024-04-27T07:50:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800b0040 con 0x7fb984033d00 2024-04-27T07:50:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb984047d10 con 0x7fb984033d00 2024-04-27T07:50:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7fb98bfff640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb984188b30 con 0x7fb984033d00 2024-04-27T07:50:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c275b640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b40e1020 0x7f79b4036c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c3e0 tx=0x7f79ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0074e0 con 0x7f79b40e1020 2024-04-27T07:50:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f70377fe640 1 --2- 172.21.15.22:0/754490002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380b3b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f70380afde0 tx=0x7f702801ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66bc573e13437d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b4189830 con 0x7f79b40e1020 2024-04-27T07:50:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b4189df0 con 0x7f79b40e1020 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f7036ffd640 1 -- 172.21.15.22:0/754490002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280ad070 con 0x7f70380d0d80 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f7036ffd640 1 -- 172.21.15.22:0/754490002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70280a7a90 con 0x7f70380d0d80 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f7036ffd640 1 -- 172.21.15.22:0/754490002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280a7e60 con 0x7f70380d0d80 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac007680 con 0x7f79b40e1020 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.168+0000 7f12483c4640 1 -- 172.21.15.22:0/1516932875 wait complete. 2024-04-27T07:50:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 Processor -- start 2024-04-27T07:50:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 -- start start 2024-04-27T07:50:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- 172.21.15.22:0/754490002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 msgr2=0x7f70380b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- 172.21.15.22:0/754490002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380b3b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f70380afde0 tx=0x7f702801ea00 comp rx=0 tx=0).stop 2024-04-27T07:50:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004a30 con 0x7f79b40e1020 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- 172.21.15.22:0/754490002 shutdown_connections 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- 172.21.15.22:0/754490002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70380ad1a0 0x7f703815ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- 172.21.15.22:0/754490002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380b3b60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- 172.21.15.22:0/754490002 >> 172.21.15.22:0/754490002 conn(0x7f70380b5670 msgr2=0x7f70380b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- 172.21.15.22:0/754490002 shutdown_connections 2024-04-27T07:50:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- 172.21.15.22:0/754490002 wait complete. 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1240070f30 0x7f124009b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f12400c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f12400c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f124006bbd0 con 0x7f124015ff20 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12401687e0 con 0x7f1240070f30 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f12483c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f124006bd50 con 0x7f124015ecf0 2024-04-27T07:50:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb98bfff640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb96c001040 con 0x7fb984033d00 2024-04-27T07:50:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f12400c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f12400c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45020/0 (socket says 172.21.15.22:45020) 2024-04-27T07:50:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 -- 172.21.15.22:0/2336279560 learned_addr learned my addr 172.21.15.22:0/2336279560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79a4001040 con 0x7f79b40e1020 2024-04-27T07:50:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246160640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1240070f30 0x7f124009b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb9800bb020 con 0x7fb984033d00 2024-04-27T07:50:59.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 Processor -- start 2024-04-27T07:50:59.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- start start 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 -- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 msgr2=0x7f12400c39a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124015ecf0 0x7f12400c39a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 -- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1240070f30 msgr2=0x7f124009b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1240070f30 0x7f124009b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124009e0b0 con 0x7f124015ff20 2024-04-27T07:50:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb978068450 0x7fb97806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb670498640 1 Processor -- start 2024-04-27T07:50:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1246961640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f12400c3ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f123801ade0 tx=0x7f12380194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7ff6b1d56640 1 Processor -- start 2024-04-27T07:50:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb670498640 1 -- start start 2024-04-27T07:50:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb980120770 con 0x7fb984033d00 2024-04-27T07:50:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb98affd640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb978068450 0x7fb97806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7fb9780706d0 0x7fb978072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9800bee50 con 0x7fb9780706d0 2024-04-27T07:50:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb980127050 con 0x7fb984033d00 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79ac011070 con 0x7f79b40e1020 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7ff6b1d56640 1 -- start start 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6b1d56640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff6ac167460 0x7ff6ac072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6b1d56640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6b1d56640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ac167880 con 0x7ff6ac162f10 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6b1d56640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ac06d500 con 0x7ff6ac167460 2024-04-27T07:50:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6b1d56640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac0d0470 con 0x7ff6ac072cb0 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb98affd640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb978068450 0x7fb97806a910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb98c074850 tx=0x7fb98c076d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12380ac040 con 0x7f124015ff20 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12380bc450 con 0x7f124015ff20 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238009510 con 0x7f124015ff20 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f12483c4640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124009e390 con 0x7f124015ff20 2024-04-27T07:50:59.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f12483c4640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124009e950 con 0x7f124015ff20 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 0x7f70381481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f703815a8c0 0x7f7038149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70380b0900 con 0x7f70380d0d80 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f703d57a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70380b1ab0 con 0x7f70380ad1a0 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f703d57a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f70380b0a80 con 0x7f703815a8c0 2024-04-27T07:50:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 Processor -- start 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.172+0000 7f79b27fc640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f79a0068450 0x7f79a006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f79ac0864a0 con 0x7f79b40e1020 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79b27fc640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f79a0070670 0x7f79a0072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79ac013e50 con 0x7f79a0070670 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79ac08d050 con 0x7f79b40e1020 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 0x7f70381481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 0x7f70381481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54640/0 (socket says 172.21.15.22:54640) 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f703cd79640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f703815a8c0 0x7f7038149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 -- 172.21.15.22:0/2644136663 learned_addr learned my addr 172.21.15.22:0/2644136663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79c1f5a640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f79a0068450 0x7f79a006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb670498640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6680b4e10 0x7fb6680b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb670498640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66815ee80 con 0x7fb66815cd00 2024-04-27T07:50:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb670498640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6680aedc0 con 0x7fb66815eaa0 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb670498640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680b57d0 con 0x7fb6680b4e10 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66e234640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fb66815eaa0 0x7fb66815c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:56314/0 (socket says 172.21.15.22:56314) 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66e234640 1 -- 172.21.15.22:0/3900023873 learned_addr learned my addr 172.21.15.22:0/3900023873 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6680b4e10 0x7fb6680b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79c1759640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f79a0070670 0x7f79a0072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb98b7fe640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7fb9780706d0 0x7fb978072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f12483c4640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1228001040 con 0x7f124015ff20 2024-04-27T07:50:59.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:52668/0 (socket says 172.21.15.22:52668) 2024-04-27T07:50:59.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 -- 172.21.15.22:0/541558443 learned_addr learned my addr 172.21.15.22:0/541558443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f70377fe640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 -- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f703815a8c0 msgr2=0x7f7038149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f703815a8c0 0x7f7038149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 -- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 msgr2=0x7f70380ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70380af400 con 0x7f70380ad1a0 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f70377fe640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70380d0d80 0x7f70380ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:59.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f703cd79640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f703815a8c0 0x7f7038149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:59.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385078503 0 0) 0x7fb6680aedc0 con 0x7fb66815eaa0 2024-04-27T07:50:59.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654005740 con 0x7fb66815eaa0 2024-04-27T07:50:59.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb98b7fe640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7fb9780706d0 0x7fb978072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 -- start start 2024-04-27T07:50:59.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fce040b3670 0x7fce040b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0409abd0 con 0x7fce04162f00 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce040b2b00 con 0x7fce040b47a0 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce0a696640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fce040b2de0 con 0x7fce040b3670 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce08e93640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fce040b3670 0x7fce040b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce09694640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fce04162f00 0x7fce040ae240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36128/0 (socket says 172.21.15.22:36128) 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79c1759640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f79a0070670 0x7f79a0072a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce09694640 1 -- 172.21.15.22:0/147666271 learned_addr learned my addr 172.21.15.22:0/147666271 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 -- 172.21.15.22:0/541558443 >> v2:172.21.15.183:3300/0 conn(0x7ff6ac167460 msgr2=0x7ff6ac072770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T07:50:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 --2- 172.21.15.22:0/541558443 >> v2:172.21.15.183:3300/0 conn(0x7ff6ac167460 0x7ff6ac072770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 -- 172.21.15.22:0/541558443 >> v1:172.21.15.183:6789/0 conn(0x7ff6ac162f10 legacy=0x7ff6ac06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7ff6abfff640 1 -- 172.21.15.22:0/541558443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac0d0750 con 0x7ff6ac072cb0 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889999606 0 0) 0x7fb66815ee80 con 0x7fb66815cd00 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6680aedc0 con 0x7fb66815cd00 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 645964223 0 0) 0x7fb654005740 con 0x7fb66815eaa0 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6680b4e10 msgr2=0x7fb6680b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6680b4e10 0x7fb6680b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 >> v1:172.21.15.22:6789/0 conn(0x7fb66815cd00 legacy=0x7fb66815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680af0a0 con 0x7fb66815eaa0 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fb660001680 con 0x7fb66815eaa0 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 >> v1:172.21.15.183:6789/0 conn(0x7fb66815eaa0 legacy=0x7fb66815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb654005d60 0x7fb654006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb654006a20 0x7fb654006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6540073e0 0x7fb6540147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb65400bbd0 con 0x7fb654006a20 2024-04-27T07:50:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb65400bd80 con 0x7fb654005d60 2024-04-27T07:50:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb65400bf80 con 0x7fb6540073e0 2024-04-27T07:50:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66ea35640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6540073e0 0x7fb6540147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== osd.1 v2:172.21.15.22:6819/73795669 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9800bee50 con 0x7fb9780706d0 2024-04-27T07:50:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f79c1f5a640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f79a0068450 0x7f79a006a910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79bc076380 tx=0x7f79bc076d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f12380af070 con 0x7f124015ff20 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1224068380 0x7f122406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1238121250 con 0x7f124015ff20 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb654006a20 0x7fb654006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66e234640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb654005d60 0x7fb654006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce08e93640 1 -- 172.21.15.22:0/147666271 >> v1:172.21.15.183:6789/0 conn(0x7fce040b47a0 legacy=0x7fce0406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce08e93640 1 -- 172.21.15.22:0/147666271 >> v1:172.21.15.22:6789/0 conn(0x7fce04162f00 legacy=0x7fce040ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fce08e93640 1 -- 172.21.15.22:0/147666271 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0409ae70 con 0x7fce040b3670 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 -- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6540073e0 msgr2=0x7fb6540147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6540073e0 0x7fb6540147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 -- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb654005d60 msgr2=0x7fb654006180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb654005d60 0x7fb654006180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66da33640 1 -- 172.21.15.22:0/3900023873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb654005740 con 0x7fb654006a20 2024-04-27T07:50:59.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7fb66ea35640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6540073e0 0x7fb6540147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f12240705a0 0x7f12240729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12380020c0 con 0x7f12240705a0 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1238121640 con 0x7f124015ff20 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f124595f640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f12240705a0 0x7f12240729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f1246160640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1224068380 0x7f122406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f124595f640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f12240705a0 0x7f12240729c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.176+0000 7f7037fff640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 0x7f70381481e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b420 tx=0x7f702c00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f1246160640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1224068380 0x7f122406a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f123c009af0 tx=0x7f123c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6abfff640 1 --2- 172.21.15.22:0/541558443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009e50 tx=0x7ff6a401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119449e9f6b06f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c011830 con 0x7f70380ad1a0 2024-04-27T07:50:59.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c016070 con 0x7f70380ad1a0 2024-04-27T07:50:59.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c011c30 con 0x7f70380ad1a0 2024-04-27T07:50:59.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== osd.5 v2:172.21.15.183:6816/2484095365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79ac013e50 con 0x7f79a0070670 2024-04-27T07:50:59.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce08e93640 1 --2- 172.21.15.22:0/147666271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b3670 0x7fce040b4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009d90 tx=0x7fcdfc01c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad46b3f4c488e11a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6aa7fc640 1 -- 172.21.15.22:0/541558443 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40a5690 con 0x7ff6ac072cb0 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6aa7fc640 1 -- 172.21.15.22:0/541558443 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a40a5830 con 0x7ff6ac072cb0 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6aa7fc640 1 -- 172.21.15.22:0/541558443 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40a5c00 con 0x7ff6ac072cb0 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 -- 172.21.15.22:0/541558443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 msgr2=0x7ff6ac06e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/541558443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009e50 tx=0x7ff6a401ca10 comp rx=0 tx=0).stop 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fcdfbfff640 1 -- 172.21.15.22:0/147666271 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0b2070 con 0x7fce040b3670 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fcdfbfff640 1 -- 172.21.15.22:0/147666271 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0a5870 con 0x7fce040b3670 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fcdfbfff640 1 -- 172.21.15.22:0/147666271 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0a5c40 con 0x7fce040b3670 2024-04-27T07:50:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 -- 172.21.15.22:0/147666271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b3670 msgr2=0x7fce040b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 --2- 172.21.15.22:0/147666271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b3670 0x7fce040b4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009d90 tx=0x7fcdfc01c8a0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fb66da33640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb654006a20 0x7fb654006ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b6810 tx=0x7fb66401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c41fade6e8f357 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== osd.2 v2:172.21.15.22:6808/2159087863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12380020c0 con 0x7f12240705a0 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 -- 172.21.15.22:0/541558443 shutdown_connections 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/541558443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac06e7d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/541558443 >> v2:172.21.15.183:3300/0 conn(0x7ff6ac167460 0x7ff6ac072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 -- 172.21.15.22:0/541558443 >> 172.21.15.22:0/541558443 conn(0x7ff6ac0d0ae0 msgr2=0x7ff6ac0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 -- 172.21.15.22:0/147666271 shutdown_connections 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 --2- 172.21.15.22:0/147666271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b3670 0x7fce040b4260 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009d90 tx=0x7fcdfc01c8a0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 -- 172.21.15.22:0/147666271 >> 172.21.15.22:0/147666271 conn(0x7fce04073450 msgr2=0x7fce040b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 -- 172.21.15.22:0/541558443 shutdown_connections 2024-04-27T07:50:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6640b2070 con 0x7fb654006a20 2024-04-27T07:50:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6640a5bf0 con 0x7fb654006a20 2024-04-27T07:50:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fb66d232640 1 -- 172.21.15.22:0/3900023873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6640b2070 con 0x7fb654006a20 2024-04-27T07:50:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f703d57a640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70380ad840 con 0x7f70380ad1a0 2024-04-27T07:50:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7f703d57a640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7038148950 con 0x7f70380ad1a0 2024-04-27T07:50:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 -- 172.21.15.22:0/147666271 shutdown_connections 2024-04-27T07:50:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- 172.21.15.22:0/3900023873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb654006a20 msgr2=0x7fb654006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 --2- 172.21.15.22:0/3900023873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb654006a20 0x7fb654006ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b6810 tx=0x7fb66401cba0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb66ea35640 1 -- 172.21.15.22:0/3900023873 reap_dead start 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- 172.21.15.22:0/3900023873 shutdown_connections 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- 172.21.15.22:0/3900023873 >> 172.21.15.22:0/3900023873 conn(0x7fb6680b6860 msgr2=0x7fb6680b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- 172.21.15.22:0/3900023873 shutdown_connections 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- 172.21.15.22:0/3900023873 wait complete. 2024-04-27T07:50:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f703d57a640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70380ad660 con 0x7f70380ad1a0 2024-04-27T07:50:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7fce0a696640 1 -- 172.21.15.22:0/147666271 wait complete. 2024-04-27T07:50:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.180+0000 7ff6b1d56640 1 -- 172.21.15.22:0/541558443 wait complete. 2024-04-27T07:50:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 Processor -- start 2024-04-27T07:50:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 Processor -- start 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 -- start start 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac1645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6ac162f10 0x7ff6ac164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6ac167460 0x7ff6ac204390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0d0190 con 0x7ff6ac162f10 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0d0010 con 0x7ff6ac167460 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6b1d56640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac16fdf0 con 0x7ff6ac072cb0 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac1645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac1645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:52690/0 (socket says 172.21.15.22:52690) 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 -- 172.21.15.22:0/594474171 learned_addr learned my addr 172.21.15.22:0/594474171 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 -- start start 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b47a0 0x7fce040ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 0x7fce0414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce040aead0 0x7fce040aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce04170c00 con 0x7fce04162f00 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce041632e0 con 0x7fce040b47a0 2024-04-27T07:50:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce0a696640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce04170d80 con 0x7fce040aead0 2024-04-27T07:50:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 Processor -- start 2024-04-27T07:50:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6abfff640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6ac167460 0x7ff6ac204390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f702c035a60 con 0x7f70380ad1a0 2024-04-27T07:50:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 0x7fce0414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 0x7fce0414c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45044/0 (socket says 172.21.15.22:45044) 2024-04-27T07:50:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 -- 172.21.15.22:0/2046401744 learned_addr learned my addr 172.21.15.22:0/2046401744 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce09694640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b47a0 0x7fce040ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce09e95640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce040aead0 0x7fce040aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 -- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6ac167460 msgr2=0x7ff6ac204390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6ac167460 0x7ff6ac204390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 -- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6ac162f10 msgr2=0x7ff6ac164ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6ac162f10 0x7ff6ac164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac159f20 con 0x7ff6ac072cb0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7ff6ab7fe640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac1645a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b440 tx=0x7ff6a000b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0015020 con 0x7ff6ac072cb0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a0011890 con 0x7ff6ac072cb0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0011c80 con 0x7ff6ac072cb0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac15a200 con 0x7ff6ac072cb0 2024-04-27T07:50:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac15a7c0 con 0x7ff6ac072cb0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7024068400 0x7f702406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f702c0862d0 con 0x7f70380ad1a0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f7024070620 0x7f7024072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f702c01dec0 con 0x7f7024070620 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f702c086680 con 0x7f70380ad1a0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7f703cd79640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f7024070620 0x7f7024072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7f70377fe640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7024068400 0x7f702406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7f70377fe640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7024068400 0x7f702406a8c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70280b2000 tx=0x7f7028008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 -- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce040aead0 msgr2=0x7fce040aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce040aead0 0x7fce040aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 -- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b47a0 msgr2=0x7fce040ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce040b47a0 0x7fce040ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fce08e93640 1 -- 172.21.15.22:0/2046401744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce040b11c0 con 0x7fce04162f00 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fce08e93640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 0x7fce0414c6e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc01cd80 tx=0x7fcdfc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0bf070 con 0x7fce04162f00 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0b3040 con 0x7fce04162f00 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce04154670 con 0x7fce04162f00 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0b2050 con 0x7fce04162f00 2024-04-27T07:50:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04154c30 con 0x7fce04162f00 2024-04-27T07:50:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.184+0000 7fb670498640 1 -- start start 2024-04-27T07:50:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb668151150 0x7fb66814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7f703cd79640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f7024070620 0x7f7024072a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668151530 0x7fb66814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb66814c980 0x7fb66814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680b4500 con 0x7fb668151150 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680af660 con 0x7fb668151530 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66e234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb668151150 0x7fb66814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66e234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb668151150 0x7fb66814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45050/0 (socket says 172.21.15.22:45050) 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66e234640 1 -- 172.21.15.22:0/175328675 learned_addr learned my addr 172.21.15.22:0/175328675 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680b4680 con 0x7fb66814c980 2024-04-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668151530 0x7fb66814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66ea35640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb66814c980 0x7fb66814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== osd.3 v2:172.21.15.22:6816/427337427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f702c01dec0 con 0x7f7024070620 2024-04-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 -- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb66814c980 msgr2=0x7fb66814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb66814c980 0x7fb66814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 -- 172.21.15.22:0/175328675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb668151150 msgr2=0x7fb66814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb668151150 0x7fb66814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66814e580 con 0x7fb668151530 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb66da33640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668151530 0x7fb66814deb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6640ac000 tx=0x7fb664009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcdfc0ad650 con 0x7fce04162f00 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcde8068380 0x7fcde806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcdfc120d80 con 0x7fce04162f00 2024-04-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6640b2070 con 0x7fb668151530 2024-04-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.188+0000 7fce09694640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcde8068380 0x7fcde806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6640be070 con 0x7fb668151530 2024-04-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664009bb0 con 0x7fb668151530 2024-04-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66814e860 con 0x7fb668151530 2024-04-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66814ee20 con 0x7fb668151530 2024-04-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff68c001040 con 0x7ff6ac072cb0 2024-04-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff6a0018020 con 0x7ff6ac072cb0 2024-04-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fce09694640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcde8068380 0x7fcde806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce00005890 tx=0x7fce00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff698068380 0x7ff69806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff6a0086ab0 con 0x7ff6ac072cb0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7ff698070520 0x7ff698072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6a0014e70 con 0x7ff698070520 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff6a008b2c0 con 0x7ff6ac072cb0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff6aaffd640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff698068380 0x7ff69806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff6abfff640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7ff698070520 0x7ff698072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff6abfff640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7ff698070520 0x7ff698072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff6aaffd640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff698068380 0x7ff69806a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac165590 tx=0x7ff69c006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== osd.0 v2:172.21.15.22:6800/2532053972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6a0014e70 con 0x7ff698070520 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.192+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb6640c2410 con 0x7fb668151530 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.196+0000 7fb65effd640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb64c0683b0 0x7fb64c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.196+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6641211b0 con 0x7fb668151530 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.196+0000 7fb66e234640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb64c0683b0 0x7fb64c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.196+0000 7fb66e234640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb64c0683b0 0x7fb64c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb660001060 tx=0x7fb660000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fb670498640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7fb650001660 0x7fb650003b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb650006bc0 con 0x7fb650001660 2024-04-27T07:50:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fce0a696640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7fce040625f0 0x7fce040629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 --> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce04154e30 con 0x7fce040625f0 2024-04-27T07:50:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fb66ea35640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7fb650001660 0x7fb650003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fce09e95640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7fce040625f0 0x7fce040629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fb66ea35640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7fb650001660 0x7fb650003b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fce09e95640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7fce040625f0 0x7fce040629d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== osd.7 v2:172.21.15.183:6824/1208358952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb650006bc0 con 0x7fb650001660 2024-04-27T07:50:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.200+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== osd.4 v2:172.21.15.183:6800/1968327270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce04154e30 con 0x7fce040625f0 2024-04-27T07:50:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 Processor -- start 2024-04-27T07:50:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb98bfff640 1 -- 172.21.15.22:0/3121092839 --> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb96c002de0 con 0x7fb9780706d0 2024-04-27T07:50:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 -- start start 2024-04-27T07:50:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb973fff640 1 -- 172.21.15.22:0/3121092839 <== osd.1 v2:172.21.15.22:6819/73795669 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb96c002de0 con 0x7fb9780706d0 2024-04-27T07:50:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc944035e90 0x7fc944055710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7fb9780706d0 msgr2=0x7fb978072af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7fb9780706d0 0x7fb978072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc951883640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc944035e90 0x7fc944055710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc951883640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc944035e90 0x7fc944055710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45062/0 (socket says 172.21.15.22:45062) 2024-04-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc951883640 1 -- 172.21.15.22:0/468555435 learned_addr learned my addr 172.21.15.22:0/468555435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944036270 0x7fc944008d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb978068450 msgr2=0x7fb97806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb978068450 0x7fb97806a910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb98c074850 tx=0x7fb98c076d60 comp rx=0 tx=0).stop 2024-04-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 msgr2=0x7fb9840e18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb971ffb640 1 --2- 172.21.15.22:0/3121092839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb984033d00 0x7fb9840e18c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb98001a9f0 tx=0x7fb9800190e0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fb98b7fe640 1 -- 172.21.15.22:0/3121092839 reap_dead start 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94403a9d0 con 0x7fc944036900 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.204+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc944053380 con 0x7fc944035e90 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc94403ac40 con 0x7fc944036270 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951082640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944036270 0x7fc944008d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc950881640 1 -- 172.21.15.22:0/468555435 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368150636 0 0) 0x7fc94403a9d0 con 0x7fc944036900 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951883640 1 -- 172.21.15.22:0/468555435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944036270 msgr2=0x7fc944008d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951883640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944036270 0x7fc944008d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951883640 1 -- 172.21.15.22:0/468555435 >> v1:172.21.15.22:6789/0 conn(0x7fc944036900 legacy=0x7fc9440e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951883640 1 -- 172.21.15.22:0/468555435 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94403af20 con 0x7fc944035e90 2024-04-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 shutdown_connections 2024-04-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 >> 172.21.15.22:0/3121092839 conn(0x7fb984009fc0 msgr2=0x7fb9840386e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 shutdown_connections 2024-04-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fb971ffb640 1 -- 172.21.15.22:0/3121092839 wait complete. 2024-04-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 --> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff68c002de0 con 0x7ff698070520 2024-04-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 --> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79a4002de0 con 0x7f79a0070670 2024-04-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.208+0000 7fc951883640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944035e90 0x7fc944055710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc948009870 tx=0x7fc94801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899d4434415eabbb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff693fff640 1 -- 172.21.15.22:0/594474171 <== osd.0 v2:172.21.15.22:6800/2532053972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff68c002de0 con 0x7ff698070520 2024-04-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc950881640 1 -- 172.21.15.22:0/468555435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480ab610 con 0x7fc944035e90 2024-04-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc950881640 1 -- 172.21.15.22:0/468555435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9480b0070 con 0x7fc944035e90 2024-04-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc950881640 1 -- 172.21.15.22:0/468555435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480ab9d0 con 0x7fc944035e90 2024-04-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7f79b27fc640 1 -- 172.21.15.22:0/3814103389 <== osd.5 v2:172.21.15.183:6816/2484095365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79a4002de0 con 0x7f79a0070670 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944035e90 msgr2=0x7fc944055710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944035e90 0x7fc944055710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc948009870 tx=0x7fc94801c580 comp rx=0 tx=0).stop 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 shutdown_connections 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944036270 0x7fc944008d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 --2- 172.21.15.22:0/468555435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944035e90 0x7fc944055710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 >> 172.21.15.22:0/468555435 conn(0x7fc94401ef50 msgr2=0x7fc944055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 shutdown_connections 2024-04-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- 172.21.15.22:0/468555435 wait complete. 2024-04-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7ff698070520 msgr2=0x7ff698072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7ff698070520 0x7ff698072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7f703d57a640 1 -- 172.21.15.22:0/2644136663 --> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f703804fea0 con 0x7f7024070620 2024-04-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 Processor -- start 2024-04-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7fc952885640 1 -- start start 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7f7034ff9640 1 -- 172.21.15.22:0/2644136663 <== osd.3 v2:172.21.15.22:6816/427337427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f703804fea0 con 0x7f7024070620 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff698068380 msgr2=0x7ff69806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff698068380 0x7ff69806a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac165590 tx=0x7ff69c006bb0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 msgr2=0x7ff6ac1645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 --2- 172.21.15.22:0/594474171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6ac072cb0 0x7ff6ac1645a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b440 tx=0x7ff6a000b900 comp rx=0 tx=0).stop 2024-04-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6abfff640 1 -- 172.21.15.22:0/594474171 reap_dead start 2024-04-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 shutdown_connections 2024-04-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 >> 172.21.15.22:0/594474171 conn(0x7ff6ac0d0ae0 msgr2=0x7ff6ac0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 shutdown_connections 2024-04-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.212+0000 7ff6b1d56640 1 -- 172.21.15.22:0/594474171 wait complete. 2024-04-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f7024070620 msgr2=0x7f7024072a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f79a0070670 msgr2=0x7f79a0072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f79a0070670 0x7f79a0072a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f7024070620 0x7f7024072a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f79a0068450 msgr2=0x7f79a006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f79a0068450 0x7f79a006a910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79bc076380 tx=0x7f79bc076d60 comp rx=0 tx=0).stop 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b40e1020 msgr2=0x7f79b4036c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 --2- 172.21.15.22:0/3814103389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79b40e1020 0x7f79b4036c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c3e0 tx=0x7f79ac00c8a0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c275b640 1 -- 172.21.15.22:0/3814103389 reap_dead start 2024-04-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944035e90 0x7fc944007980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 shutdown_connections 2024-04-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 >> 172.21.15.22:0/3814103389 conn(0x7f79b4009fc0 msgr2=0x7f79b4053a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc944036270 0x7fc94404ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 shutdown_connections 2024-04-27T07:50:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f79c2f5c640 1 -- 172.21.15.22:0/3814103389 wait complete. 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944036900 0x7fc9440204b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc94403b8a0 con 0x7fc944036900 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc94403b720 con 0x7fc944036270 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc9440f02c0 con 0x7fc944035e90 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc951082640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc944036270 0x7fc94404ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944036900 0x7fc9440204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc951082640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc944036270 0x7fc94404ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54686/0 (socket says 172.21.15.22:54686) 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc951082640 1 -- 172.21.15.22:0/1302534222 learned_addr learned my addr 172.21.15.22:0/1302534222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc951883640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944035e90 0x7fc944007980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7024068400 msgr2=0x7f702406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 -- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944035e90 msgr2=0x7fc944007980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944035e90 0x7fc944007980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 -- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc944036270 msgr2=0x7fc94404ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc944036270 0x7fc94404ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc944049be0 con 0x7fc944036900 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc951883640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc944035e90 0x7fc944007980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7024068400 0x7f702406a8c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70280b2000 tx=0x7f7028008040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952084640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944036900 0x7fc9440204b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc94c06aa20 tx=0x7fc94c0750d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 msgr2=0x7f70381481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 --2- 172.21.15.22:0/2644136663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70380ad1a0 0x7f70381481e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b420 tx=0x7f702c00b8e0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c075730 con 0x7fc944036900 2024-04-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c0758d0 con 0x7fc944036900 2024-04-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c093400 con 0x7fc944036900 2024-04-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f703cd79640 1 -- 172.21.15.22:0/2644136663 reap_dead start 2024-04-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94404a1a0 con 0x7fc944036900 2024-04-27T07:50:59.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 shutdown_connections 2024-04-27T07:50:59.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 >> 172.21.15.22:0/2644136663 conn(0x7f70380b5670 msgr2=0x7f703806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.216+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94404a760 con 0x7fc944036900 2024-04-27T07:50:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.220+0000 7f12483c4640 1 -- 172.21.15.22:0/2336279560 --> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1228002d60 con 0x7f12240705a0 2024-04-27T07:50:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.220+0000 7f1236ffd640 1 -- 172.21.15.22:0/2336279560 <== osd.2 v2:172.21.15.22:6808/2159087863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1228002d60 con 0x7f12240705a0 2024-04-27T07:50:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 shutdown_connections 2024-04-27T07:50:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.220+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9440057f0 con 0x7fc944036900 2024-04-27T07:50:59.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc94c0935a0 con 0x7fc944036900 2024-04-27T07:50:59.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7f701e7fc640 1 -- 172.21.15.22:0/2644136663 wait complete. 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc930068380 0x7fc93006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc94c106060 con 0x7fc944036900 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7fc9300705a0 0x7fc9300729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc94c091e80 con 0x7fc9300705a0 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc94c1014d0 con 0x7fc944036900 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc951883640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc930068380 0x7fc93006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc951082640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7fc9300705a0 0x7fc9300729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc951883640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc930068380 0x7fc93006a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc9480af000 tx=0x7fc9480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fc951082640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7fc9300705a0 0x7fc9300729c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.224+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 --> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce0404fea0 con 0x7fce040625f0 2024-04-27T07:50:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f12240705a0 msgr2=0x7f12240729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f12240705a0 0x7f12240729c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7fcdf9ffb640 1 -- 172.21.15.22:0/2046401744 <== osd.4 v2:172.21.15.183:6800/1968327270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce0404fea0 con 0x7fce040625f0 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1224068380 msgr2=0x7f122406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1224068380 0x7f122406a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f123c009af0 tx=0x7f123c008040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 msgr2=0x7f12400c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 --2- 172.21.15.22:0/2336279560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f124015ff20 0x7f12400c3ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f123801ade0 tx=0x7f12380194d0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1246961640 1 -- 172.21.15.22:0/2336279560 reap_dead start 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 shutdown_connections 2024-04-27T07:50:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 >> 172.21.15.22:0/2336279560 conn(0x7f12400b3020 msgr2=0x7f12400afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== osd.6 v2:172.21.15.183:6808/3842666624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc94c091e80 con 0x7fc9300705a0 2024-04-27T07:50:59.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 shutdown_connections 2024-04-27T07:50:59.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.228+0000 7f1234ff9640 1 -- 172.21.15.22:0/2336279560 wait complete. 2024-04-27T07:50:59.265 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T07:50:59.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.236+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 --> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb650005c50 con 0x7fb650001660 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7fce040625f0 msgr2=0x7fce040629d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7fce040625f0 0x7fce040629d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcde8068380 msgr2=0x7fcde806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcde8068380 0x7fcde806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce00005890 tx=0x7fce00009040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 msgr2=0x7fce0414c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 --2- 172.21.15.22:0/2046401744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce04162f00 0x7fce0414c6e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc01cd80 tx=0x7fcdfc019640 comp rx=0 tx=0).stop 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce09e95640 1 -- 172.21.15.22:0/2046401744 reap_dead start 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 shutdown_connections 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 >> 172.21.15.22:0/2046401744 conn(0x7fce04073450 msgr2=0x7fce0415f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 shutdown_connections 2024-04-27T07:50:59.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fce0a696640 1 -- 172.21.15.22:0/2046401744 wait complete. 2024-04-27T07:50:59.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb65effd640 1 -- 172.21.15.22:0/175328675 <== osd.7 v2:172.21.15.183:6824/1208358952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb650005c50 con 0x7fb650001660 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7fb650001660 msgr2=0x7fb650003b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7fb650001660 0x7fb650003b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb64c0683b0 msgr2=0x7fb64c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb64c0683b0 0x7fb64c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb660001060 tx=0x7fb660000f90 comp rx=0 tx=0).stop 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668151530 msgr2=0x7fb66814deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 --2- 172.21.15.22:0/175328675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb668151530 0x7fb66814deb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6640ac000 tx=0x7fb664009750 comp rx=0 tx=0).stop 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb66ea35640 1 -- 172.21.15.22:0/175328675 reap_dead start 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 shutdown_connections 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 >> 172.21.15.22:0/175328675 conn(0x7fb6680b6860 msgr2=0x7fb668158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 shutdown_connections 2024-04-27T07:50:59.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.240+0000 7fb670498640 1 -- 172.21.15.22:0/175328675 wait complete. 2024-04-27T07:50:59.270 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T07:50:59.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.244+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 --> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9440057f0 con 0x7fc9300705a0 2024-04-27T07:50:59.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc9427fc640 1 -- 172.21.15.22:0/1302534222 <== osd.6 v2:172.21.15.183:6808/3842666624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc9440057f0 con 0x7fc9300705a0 2024-04-27T07:50:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-27T07:50:59.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7fc9300705a0 msgr2=0x7fc9300729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7fc9300705a0 0x7fc9300729c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc930068380 msgr2=0x7fc93006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc930068380 0x7fc93006a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc9480af000 tx=0x7fc9480a7000 comp rx=0 tx=0).stop 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944036900 msgr2=0x7fc9440204b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 --2- 172.21.15.22:0/1302534222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc944036900 0x7fc9440204b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc94c06aa20 tx=0x7fc94c0750d0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952084640 1 -- 172.21.15.22:0/1302534222 reap_dead start 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 shutdown_connections 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 >> 172.21.15.22:0/1302534222 conn(0x7fc94401ef50 msgr2=0x7fc94400c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 shutdown_connections 2024-04-27T07:50:59.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.248+0000 7fc952885640 1 -- 172.21.15.22:0/1302534222 wait complete. 2024-04-27T07:50:59.275 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T07:50:59.276 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T07:50:59.277 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-27T07:50:59.278 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-04-27T07:50:59.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-27T07:50:59.279 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-27T07:50:59.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.480+0000 7fc9596d6640 1 Processor -- start 2024-04-27T07:50:59.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 -- start start 2024-04-27T07:50:59.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.480+0000 7fca90532640 1 Processor -- start 2024-04-27T07:50:59.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.480+0000 7fca90532640 1 -- start start 2024-04-27T07:50:59.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 Processor -- start 2024-04-27T07:50:59.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca90532640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca80054230 0x7fca8004f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 -- start start 2024-04-27T07:50:59.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 0x7fc95415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9540cc770 con 0x7fc9540cbf90 2024-04-27T07:50:59.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc9540cbbf0 con 0x7fc9540d0ce0 2024-04-27T07:50:59.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9596d6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc9540cb2c0 con 0x7fc954073930 2024-04-27T07:50:59.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca90532640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fca80007290 0x7fca8004f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca90532640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca80053800 con 0x7fca80054610 2024-04-27T07:50:59.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca90532640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fca80053ae0 con 0x7fca80054230 2024-04-27T07:50:59.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca90532640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fca80037b30 con 0x7fca80007290 2024-04-27T07:50:59.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 Processor -- start 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc952ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 0x7fc95415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca80054230 0x7fca8004f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca80054230 0x7fca8004f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45074/0 (socket says 172.21.15.22:45074) 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 -- 172.21.15.22:0/3401486255 learned_addr learned my addr 172.21.15.22:0/3401486255 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc952ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 0x7fc95415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45086/0 (socket says 172.21.15.22:45086) 2024-04-27T07:50:59.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc952ffd640 1 -- 172.21.15.22:0/1453770961 learned_addr learned my addr 172.21.15.22:0/1453770961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8eacf640 1 --2- 172.21.15.22:0/3401486255 >> v2:172.21.15.183:3300/0 conn(0x7fca80007290 0x7fca8004f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9537fe640 1 --2- 172.21.15.22:0/1453770961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faf38072270 0x7faf3816e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf380726c0 con 0x7faf38071dc0 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3806e8e0 con 0x7faf3815d0a0 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e952640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7faf3806ebc0 con 0x7faf38072270 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3d14f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7faf38071dc0 0x7faf3816aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36150/0 (socket says 172.21.15.22:36150) 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3d14f640 1 -- 172.21.15.22:0/653855034 learned_addr learned my addr 172.21.15.22:0/653855034 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3e151640 1 --2- 172.21.15.22:0/653855034 >> v2:172.21.15.183:3300/0 conn(0x7faf38072270 0x7faf3816e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9537fe640 1 -- 172.21.15.22:0/1453770961 >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 msgr2=0x7fc95415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9537fe640 1 --2- 172.21.15.22:0/1453770961 >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 0x7fc95415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9537fe640 1 -- 172.21.15.22:0/1453770961 >> v1:172.21.15.183:6789/0 conn(0x7fc9540cbf90 legacy=0x7fc954073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fc9537fe640 1 -- 172.21.15.22:0/1453770961 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9540cb5a0 con 0x7fc954073930 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756157463 0 0) 0x7faf380726c0 con 0x7faf38071dc0 2024-04-27T07:50:59.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf24003530 con 0x7faf38071dc0 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 -- start start 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f600b3bf0 0x7f2f600b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2f600b0c10 0x7f2f600ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f600ad4f0 con 0x7f2f60073d00 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2f600b4510 con 0x7f2f600b3bf0 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f64c12640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2f600b1460 con 0x7f2f600b0c10 2024-04-27T07:50:59.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 367524288 0 0) 0x7faf24003530 con 0x7faf38071dc0 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 >> v2:172.21.15.183:3300/0 conn(0x7faf38072270 msgr2=0x7faf3816e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 --2- 172.21.15.22:0/653855034 >> v2:172.21.15.183:3300/0 conn(0x7faf38072270 0x7faf3816e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 >> v1:172.21.15.183:6789/0 conn(0x7faf3815d0a0 legacy=0x7faf3806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3806eea0 con 0x7faf38071dc0 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7faf30001a00 con 0x7faf38071dc0 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 >> v1:172.21.15.22:6789/0 conn(0x7faf38071dc0 legacy=0x7faf3816aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf24003a30 0x7faf24003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf24005220 0x7faf2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f5f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f600b3bf0 0x7f2f600b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f5f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f600b3bf0 0x7f2f600b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45092/0 (socket says 172.21.15.22:45092) 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/1256885296 learned_addr learned my addr 172.21.15.22:0/1256885296 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7f2f5ffff640 1 --2- 172.21.15.22:0/1256885296 >> v2:172.21.15.183:3300/0 conn(0x7f2f600b0c10 0x7f2f600ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf3806ebc0 con 0x7faf24005220 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf2400c170 con 0x7faf24003a30 2024-04-27T07:50:59.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7faf2400c370 con 0x7faf24004810 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7faf3d14f640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/1256885296 >> v2:172.21.15.183:3300/0 conn(0x7f2f600b0c10 msgr2=0x7f2f600ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5f7fe640 1 --2- 172.21.15.22:0/1256885296 >> v2:172.21.15.183:3300/0 conn(0x7f2f600b0c10 0x7f2f600ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/1256885296 >> v1:172.21.15.183:6789/0 conn(0x7f2f60073d00 legacy=0x7f2f600ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 -- 172.21.15.22:0/3401486255 >> v2:172.21.15.183:3300/0 conn(0x7fca80007290 msgr2=0x7fca8004f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 --2- 172.21.15.22:0/3401486255 >> v2:172.21.15.183:3300/0 conn(0x7fca80007290 0x7fca8004f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 -- 172.21.15.22:0/3401486255 >> v1:172.21.15.183:6789/0 conn(0x7fca80054610 legacy=0x7fca80104850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.484+0000 7fca8e2ce640 1 -- 172.21.15.22:0/3401486255 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8003a550 con 0x7fca80054230 2024-04-27T07:50:59.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/1256885296 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f600ad760 con 0x7f2f600b3bf0 2024-04-27T07:50:59.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3e151640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf24005220 0x7faf2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d950640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf24003a30 0x7faf24003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d14f640 1 -- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf24003a30 msgr2=0x7faf24003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d14f640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf24003a30 0x7faf24003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d14f640 1 -- 172.21.15.22:0/653855034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf24005220 msgr2=0x7faf2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d14f640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf24005220 0x7faf2400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3d14f640 1 -- 172.21.15.22:0/653855034 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3806eea0 con 0x7faf24004810 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3e151640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf24005220 0x7faf2400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7faf3e151640 1 -- 172.21.15.22:0/653855034 reap_dead start 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca8e2ce640 1 --2- 172.21.15.22:0/3401486255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80054230 0x7fca8004f3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca88069fa0 tx=0x7fca88089950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63904c779ed8ef6a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f28428bf640 1 Processor -- start 2024-04-27T07:50:59.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f28428bf640 1 -- start start 2024-04-27T07:50:59.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca8d2cc640 1 -- 172.21.15.22:0/3401486255 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88118070 con 0x7fca80054230 2024-04-27T07:50:59.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca8d2cc640 1 -- 172.21.15.22:0/3401486255 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca88113760 con 0x7fca80054230 2024-04-27T07:50:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9537fe640 1 --2- 172.21.15.22:0/1453770961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540ad210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9480097c0 tx=0x7fc94801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ac093e2c77ba94 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc951ffb640 1 -- 172.21.15.22:0/1453770961 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b0070 con 0x7fc954073930 2024-04-27T07:50:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca8d2cc640 1 -- 172.21.15.22:0/3401486255 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88113b90 con 0x7fca80054230 2024-04-27T07:50:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 -- 172.21.15.22:0/3401486255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80054230 msgr2=0x7fca8004f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 --2- 172.21.15.22:0/3401486255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80054230 0x7fca8004f3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca88069fa0 tx=0x7fca88089950 comp rx=0 tx=0).stop 2024-04-27T07:50:59.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7ffb72e05640 1 Processor -- start 2024-04-27T07:50:59.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7ffb72e05640 1 -- start start 2024-04-27T07:50:59.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 -- 172.21.15.22:0/3401486255 shutdown_connections 2024-04-27T07:50:59.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 --2- 172.21.15.22:0/3401486255 >> v2:172.21.15.183:3300/0 conn(0x7fca80007290 0x7fca8004f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 --2- 172.21.15.22:0/3401486255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80054230 0x7fca8004f3e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 -- 172.21.15.22:0/3401486255 >> 172.21.15.22:0/3401486255 conn(0x7fca80009fc0 msgr2=0x7fca8000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f28428bf640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28340df390 0x7f2834007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f283bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f283404fa80 0x7f283404fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36158/0 (socket says 172.21.15.22:36158) 2024-04-27T07:50:59.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc951ffb640 1 -- 172.21.15.22:0/1453770961 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9480ab600 con 0x7fc954073930 2024-04-27T07:50:59.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc951ffb640 1 -- 172.21.15.22:0/1453770961 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480ab9d0 con 0x7fc954073930 2024-04-27T07:50:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 -- 172.21.15.22:0/1453770961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 msgr2=0x7fc9540ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 --2- 172.21.15.22:0/1453770961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540ad210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9480097c0 tx=0x7fc94801c350 comp rx=0 tx=0).stop 2024-04-27T07:50:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 -- 172.21.15.22:0/3401486255 shutdown_connections 2024-04-27T07:50:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fce3d186640 1 Processor -- start 2024-04-27T07:50:59.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5f7fe640 1 --2- 172.21.15.22:0/1256885296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f600b3bf0 0x7f2f600b3fd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f60072e10 tx=0x7f2f600af6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4668d351fe715a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 -- 172.21.15.22:0/1453770961 shutdown_connections 2024-04-27T07:50:59.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 --2- 172.21.15.22:0/1453770961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540ad210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 --2- 172.21.15.22:0/1453770961 >> v2:172.21.15.22:3300/0 conn(0x7fc9540d0ce0 0x7fc95415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 -- 172.21.15.22:0/1453770961 >> 172.21.15.22:0/1453770961 conn(0x7fc9540d01c0 msgr2=0x7fc9540d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 -- 172.21.15.22:0/1453770961 shutdown_connections 2024-04-27T07:50:59.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fca90532640 1 -- 172.21.15.22:0/3401486255 wait complete. 2024-04-27T07:50:59.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5e7fc640 1 -- 172.21.15.22:0/1256885296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580ad070 con 0x7f2f600b3bf0 2024-04-27T07:50:59.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5e7fc640 1 -- 172.21.15.22:0/1256885296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f580a7e10 con 0x7f2f600b3bf0 2024-04-27T07:50:59.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f2f5e7fc640 1 -- 172.21.15.22:0/1256885296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800be60 con 0x7f2f600b3bf0 2024-04-27T07:50:59.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f283b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28340df390 0x7f2834007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f283b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28340df390 0x7f2834007190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45118/0 (socket says 172.21.15.22:45118) 2024-04-27T07:50:59.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7f283bfff640 1 -- 172.21.15.22:0/2717692916 learned_addr learned my addr 172.21.15.22:0/2717692916 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fce3d186640 1 -- start start 2024-04-27T07:50:59.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.488+0000 7fc9596d6640 1 -- 172.21.15.22:0/1453770961 wait complete. 2024-04-27T07:50:59.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3d14f640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf24004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faf3000a8f0 tx=0x7faf3001d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9e5bed18d38860 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb72e05640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffb6c09aa30 0x7ffb6c06bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb72e05640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb6c0b3ca0 con 0x7ffb6c146180 2024-04-27T07:50:59.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2834054390 con 0x7f283404fa80 2024-04-27T07:50:59.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb72e05640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb6c15e970 con 0x7ffb6c0ae240 2024-04-27T07:50:59.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb72e05640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6c15ec50 con 0x7ffb6c09aa30 2024-04-27T07:50:59.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000a4c0 con 0x7faf24004810 2024-04-27T07:50:59.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf300ae650 con 0x7faf24004810 2024-04-27T07:50:59.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3c94e640 1 -- 172.21.15.22:0/653855034 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300aea40 con 0x7faf24004810 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3d186640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fce380b4250 0x7fce380b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3d186640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce38073d00 0x7fce380ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3d186640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce380ad4f0 con 0x7fce38072300 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3d186640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fce380b1460 con 0x7fce380b4250 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3d186640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce380ad760 con 0x7fce38073d00 2024-04-27T07:50:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- 172.21.15.22:0/1256885296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f600b3bf0 msgr2=0x7f2f600b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 --2- 172.21.15.22:0/1256885296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f600b3bf0 0x7f2f600b3fd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f60072e10 tx=0x7f2f600af6a0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- 172.21.15.22:0/1256885296 shutdown_connections 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 --2- 172.21.15.22:0/1256885296 >> v2:172.21.15.183:3300/0 conn(0x7f2f600b0c10 0x7f2f600ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 --2- 172.21.15.22:0/1256885296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f600b3bf0 0x7f2f600b3fd0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f60072e10 tx=0x7f2f600af6a0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- 172.21.15.22:0/1256885296 >> 172.21.15.22:0/1256885296 conn(0x7f2f600b5890 msgr2=0x7f2f600731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce37fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fce38072300 0x7fce380b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36182/0 (socket says 172.21.15.22:36182) 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce37fff640 1 -- 172.21.15.22:0/1867270627 learned_addr learned my addr 172.21.15.22:0/1867270627 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71e03640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ffb6c146180 0x7ffb6c0af9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36172/0 (socket says 172.21.15.22:36172) 2024-04-27T07:50:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71e03640 1 -- 172.21.15.22:0/2115330139 learned_addr learned my addr 172.21.15.22:0/2115330139 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71602640 1 --2- 172.21.15.22:0/2115330139 >> v2:172.21.15.183:3300/0 conn(0x7ffb6c09aa30 0x7ffb6c06bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce377fe640 1 --2- 172.21.15.22:0/1867270627 >> v2:172.21.15.22:3300/0 conn(0x7fce380b4250 0x7fce380b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce3c985640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce38073d00 0x7fce380ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- 172.21.15.22:0/1256885296 shutdown_connections 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- 172.21.15.22:0/1256885296 wait complete. 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce36ffd640 1 -- 172.21.15.22:0/1867270627 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671551919 0 0) 0x7fce380ad4f0 con 0x7fce38072300 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce36ffd640 1 -- 172.21.15.22:0/1867270627 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce20003530 con 0x7fce38072300 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71602640 1 -- 172.21.15.22:0/2115330139 >> v1:172.21.15.183:6789/0 conn(0x7ffb6c0ae240 legacy=0x7ffb6c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71602640 1 -- 172.21.15.22:0/2115330139 >> v1:172.21.15.22:6789/0 conn(0x7ffb6c146180 legacy=0x7ffb6c0af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7ffb71602640 1 -- 172.21.15.22:0/2115330139 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c15ef30 con 0x7ffb6c09aa30 2024-04-27T07:50:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 Processor -- start 2024-04-27T07:50:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 -- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 msgr2=0x7faf24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf24004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faf3000a8f0 tx=0x7faf3001d3c0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce377fe640 1 -- 172.21.15.22:0/1867270627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce38073d00 msgr2=0x7fce380ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce377fe640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce38073d00 0x7fce380ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce377fe640 1 -- 172.21.15.22:0/1867270627 >> v1:172.21.15.22:6789/0 conn(0x7fce38072300 legacy=0x7fce380b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fce377fe640 1 -- 172.21.15.22:0/1867270627 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce380ada40 con 0x7fce380b4250 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28340060f0 con 0x7f283404f5c0 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2834005ea0 con 0x7f28340df390 2024-04-27T07:50:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 Processor -- start 2024-04-27T07:50:59.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 -- start start 2024-04-27T07:50:59.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 0x7fc954151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 0x7fc9540ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9540cc550 con 0x7fc9540d0ce0 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9540cc3d0 con 0x7fc9540cbf90 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9596d6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc954073db0 con 0x7fc954073930 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc952ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 0x7fc954151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 0x7fc954151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54752/0 (socket says 172.21.15.22:54752) 2024-04-27T07:50:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9527fc640 1 -- 172.21.15.22:0/4279148377 learned_addr learned my addr 172.21.15.22:0/4279148377 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7fc9537fe640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 0x7fc9540ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 -- 172.21.15.22:0/653855034 shutdown_connections 2024-04-27T07:50:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 --2- 172.21.15.22:0/653855034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf24004c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 -- 172.21.15.22:0/653855034 >> 172.21.15.22:0/653855034 conn(0x7faf380d01c0 msgr2=0x7faf380d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 -- 172.21.15.22:0/653855034 shutdown_connections 2024-04-27T07:50:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 -- 172.21.15.22:0/653855034 wait complete. 2024-04-27T07:50:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7faf3e952640 1 Processor -- start 2024-04-27T07:50:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f283b7fe640 1 -- 172.21.15.22:0/2717692916 >> v1:172.21.15.183:6789/0 conn(0x7f283404f5c0 legacy=0x7f2834038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f283b7fe640 1 -- 172.21.15.22:0/2717692916 >> v1:172.21.15.22:6789/0 conn(0x7f283404fa80 legacy=0x7f283404fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f283b7fe640 1 -- 172.21.15.22:0/2717692916 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28340063d0 con 0x7f28340df390 2024-04-27T07:50:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.492+0000 7f2f64c12640 1 -- start start 2024-04-27T07:50:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 0x7f2f601fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb71602640 1 --2- 172.21.15.22:0/2115330139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c06bbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097c0 tx=0x7ffb5c01c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf5b2b36f79395f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f60152090 0x7f2f601462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f600b11a0 con 0x7f2f60152090 2024-04-27T07:50:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f600b1020 con 0x7f2f600b0c10 2024-04-27T07:50:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2f60167160 con 0x7f2f60073d00 2024-04-27T07:50:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb70e01640 1 -- 172.21.15.22:0/2115330139 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b0070 con 0x7ffb6c09aa30 2024-04-27T07:50:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 -- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 msgr2=0x7fc9540999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 -- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 msgr2=0x7fc954151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 0x7fc954151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 -- 172.21.15.22:0/4279148377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc954167f30 con 0x7fc9540d0ce0 2024-04-27T07:50:59.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb70e01640 1 -- 172.21.15.22:0/2115330139 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0ab600 con 0x7ffb6c09aa30 2024-04-27T07:50:59.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb70e01640 1 -- 172.21.15.22:0/2115330139 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0aba10 con 0x7ffb6c09aa30 2024-04-27T07:50:59.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:52766/0 (socket says 172.21.15.22:52766) 2024-04-27T07:50:59.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/3248386366 learned_addr learned my addr 172.21.15.22:0/3248386366 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9537fe640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 0x7fc9540ad130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9480ab000 tx=0x7fc9480190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- start start 2024-04-27T07:50:59.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce377fe640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce380b4250 0x7fce380b0c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0097c0 tx=0x7fce2c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8ab19ff8fe003e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5effd640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 0x7f2f601fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 msgr2=0x7f2f601fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 0x7f2f601fb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/3248386366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f60152090 msgr2=0x7f2f601462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f60152090 0x7f2f601462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 -- 172.21.15.22:0/3248386366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f60148880 con 0x7f2f60073d00 2024-04-27T07:50:59.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5effd640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 0x7f2f601fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- 172.21.15.22:0/2115330139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 msgr2=0x7ffb6c06bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 --2- 172.21.15.22:0/2115330139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c06bbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097c0 tx=0x7ffb5c01c4c0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- 172.21.15.22:0/2115330139 shutdown_connections 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 --2- 172.21.15.22:0/2115330139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c06bbd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- 172.21.15.22:0/2115330139 >> 172.21.15.22:0/2115330139 conn(0x7ffb6c073450 msgr2=0x7ffb6c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fca90532640 1 Processor -- start 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf381605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 0x7faf3815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 0x7faf3815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf38163420 con 0x7faf38200a10 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf3806f460 con 0x7faf38200590 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7faf3815d480 con 0x7faf24004810 2024-04-27T07:50:59.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e151640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 0x7faf3815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e151640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 0x7faf3815ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45138/0 (socket says 172.21.15.22:45138) 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e151640 1 -- 172.21.15.22:0/442909689 learned_addr learned my addr 172.21.15.22:0/442909689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 0x7faf3815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d950640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf381605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 -- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 msgr2=0x7faf381605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf381605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 -- 172.21.15.22:0/442909689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 msgr2=0x7faf3815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 0x7faf3815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 -- 172.21.15.22:0/442909689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3809baa0 con 0x7faf38200590 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d950640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf381605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:50:59.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3d14f640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 0x7faf3815e530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c540 tx=0x7faf24005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b8020 con 0x7fc9540d0ce0 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fca90532640 1 -- start start 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f5f7fe640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f580b3040 tx=0x7f2f5801bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580c0070 con 0x7f2f60073d00 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f58007e30 con 0x7f2f60073d00 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580ad040 con 0x7f2f60073d00 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 -- 172.21.15.22:0/3248386366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f60148b60 con 0x7f2f60073d00 2024-04-27T07:50:59.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f2f64c12640 1 -- 172.21.15.22:0/3248386366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f60149120 con 0x7f2f60073d00 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f283b7fe640 1 --2- 172.21.15.22:0/2717692916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28340df390 0x7f2834007190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f283001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936539036771a83f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f283affd640 1 -- 172.21.15.22:0/2717692916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300b0070 con 0x7f28340df390 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f283affd640 1 -- 172.21.15.22:0/2717692916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2830004230 con 0x7f28340df390 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f283affd640 1 -- 172.21.15.22:0/2717692916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830003620 con 0x7f28340df390 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce36ffd640 1 -- 172.21.15.22:0/1867270627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0b2070 con 0x7fce380b4250 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce36ffd640 1 -- 172.21.15.22:0/1867270627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c0ad680 con 0x7fce380b4250 2024-04-27T07:50:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce36ffd640 1 -- 172.21.15.22:0/1867270627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0ada50 con 0x7fce380b4250 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce3d186640 1 -- 172.21.15.22:0/1867270627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce380b4250 msgr2=0x7fce380b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fce3d186640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce380b4250 0x7fce380b0c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0097c0 tx=0x7fce2c01c470 comp rx=0 tx=0).stop 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- 172.21.15.22:0/2115330139 shutdown_connections 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- 172.21.15.22:0/2115330139 wait complete. 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 Processor -- start 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f86dd5aa640 1 Processor -- start 2024-04-27T07:50:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f86dd5aa640 1 -- start start 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- start start 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb6c0ae240 0x7ffb6c0d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c16acc0 con 0x7ffb6c146180 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c06c520 con 0x7ffb6c09aa30 2024-04-27T07:50:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72e05640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6c16ae40 con 0x7ffb6c0ae240 2024-04-27T07:50:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f86dd5aa640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f86d80d16c0 0x7f86d80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7ffb72604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45150/0 (socket says 172.21.15.22:45150) 2024-04-27T07:50:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 -- 172.21.15.22:0/1375161032 learned_addr learned my addr 172.21.15.22:0/1375161032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb71e03640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7f86d7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f86d815fb90 0x7f86d80d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36184/0 (socket says 172.21.15.22:36184) 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dd5aa640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d80d0630 con 0x7f86d815fb90 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d77fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f86d8160460 0x7f86d80d0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:56404/0 (socket says 172.21.15.22:56404) 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d77fe640 1 -- 172.21.15.22:0/4286033239 learned_addr learned my addr 172.21.15.22:0/4286033239 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:50:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9480aa950 con 0x7fc9540d0ce0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9596d6640 1 -- 172.21.15.22:0/4279148377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc954152ac0 con 0x7fc9540d0ce0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9596d6640 1 -- 172.21.15.22:0/4279148377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541681d0 con 0x7fc9540d0ce0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7fc9596d6640 1 -- 172.21.15.22:0/4279148377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95404fea0 con 0x7fc9540d0ce0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b0040 con 0x7fc9540d0ce0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f86d80d16c0 0x7f86d80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28340df390 msgr2=0x7f2834007190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 --2- 172.21.15.22:0/2717692916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28340df390 0x7f2834007190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f283001c480 comp rx=0 tx=0).stop 2024-04-27T07:50:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb71602640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb6c0ae240 0x7ffb6c0d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d80d0070 con 0x7f86d8160460 2024-04-27T07:50:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f86d80d0350 con 0x7f86d80d16c0 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30001950 con 0x7faf38200590 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf300ae800 con 0x7faf38200590 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- 172.21.15.22:0/442909689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3809ca30 con 0x7faf38200590 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.496+0000 7faf3e952640 1 -- 172.21.15.22:0/442909689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3809ad30 con 0x7faf38200590 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f64c12640 1 -- 172.21.15.22:0/3248386366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6004fea0 con 0x7f2f60073d00 2024-04-27T07:50:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300ae9b0 con 0x7faf38200590 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 -- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb6c0ae240 msgr2=0x7ffb6c0d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb6c0ae240 0x7ffb6c0d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 -- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 msgr2=0x7ffb6c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 -- 172.21.15.22:0/1375161032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c0cee00 con 0x7ffb6c146180 2024-04-27T07:50:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca80054230 0x7fca800f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72604640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400c710 tx=0x7ffb6400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 -- 172.21.15.22:0/1867270627 shutdown_connections 2024-04-27T07:50:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce38073d00 0x7fce380ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 --2- 172.21.15.22:0/1867270627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce380b4250 0x7fce380b0c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 -- 172.21.15.22:0/1867270627 >> 172.21.15.22:0/1867270627 conn(0x7fce380b5890 msgr2=0x7fce380731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7faf3e952640 1 -- 172.21.15.22:0/442909689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3804fea0 con 0x7faf38200590 2024-04-27T07:50:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 -- 172.21.15.22:0/1867270627 shutdown_connections 2024-04-27T07:50:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca80054610 0x7fca8004d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca8010dd10 con 0x7fca80007290 2024-04-27T07:50:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca8010db90 con 0x7fca80054230 2024-04-27T07:50:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca90532640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fca8010de90 con 0x7fca80054610 2024-04-27T07:50:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45152/0 (socket says 172.21.15.22:45152) 2024-04-27T07:50:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 -- 172.21.15.22:0/729271303 learned_addr learned my addr 172.21.15.22:0/729271303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8eacf640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca80054610 0x7fca8004d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 shutdown_connections 2024-04-27T07:50:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 --2- 172.21.15.22:0/2717692916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28340df390 0x7f2834007190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 >> 172.21.15.22:0/2717692916 conn(0x7f2834009fc0 msgr2=0x7f283400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2f58008280 con 0x7f2f60073d00 2024-04-27T07:50:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f47fff640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 0x7f2f4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2f58120740 con 0x7f2f60073d00 2024-04-27T07:50:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 -- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca80054610 msgr2=0x7fca8004d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca80054610 0x7fca8004d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 -- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca80054230 msgr2=0x7fca800f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca80054230 0x7fca800f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982888644 0 0) 0x7f86d80d0070 con 0x7f86d8160460 2024-04-27T07:50:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86c0003530 con 0x7f86d8160460 2024-04-27T07:50:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fca8e2ce640 1 -- 172.21.15.22:0/729271303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8010b1c0 con 0x7fca80007290 2024-04-27T07:50:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 -- 172.21.15.22:0/1867270627 wait complete. 2024-04-27T07:50:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f5effd640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 0x7f2f4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 shutdown_connections 2024-04-27T07:50:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64016020 con 0x7ffb6c146180 2024-04-27T07:50:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2640005155 0 0) 0x7f86c0003530 con 0x7f86d8160460 2024-04-27T07:50:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 >> v2:172.21.15.183:3300/0 conn(0x7f86d80d16c0 msgr2=0x7f86d80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 --2- 172.21.15.22:0/4286033239 >> v2:172.21.15.183:3300/0 conn(0x7f86d80d16c0 0x7f86d80d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 >> v1:172.21.15.22:6789/0 conn(0x7f86d815fb90 legacy=0x7f86d80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d80d2250 con 0x7f86d8160460 2024-04-27T07:50:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 Processor -- start 2024-04-27T07:50:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f86c80050a0 con 0x7f86d8160460 2024-04-27T07:50:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 >> v1:172.21.15.183:6789/0 conn(0x7f86d8160460 legacy=0x7f86d80d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86c0003a30 0x7f86c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c0004810 0x7f86c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86d80d0350 con 0x7f86c0004810 2024-04-27T07:50:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86c000c100 con 0x7f86c00051d0 2024-04-27T07:50:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f86c000c300 con 0x7f86c0003a30 2024-04-27T07:50:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f28428bf640 1 -- 172.21.15.22:0/2717692916 wait complete. 2024-04-27T07:50:59.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 -- start start 2024-04-27T07:50:59.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f2f5effd640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 0x7f2f4c06a840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f600b4240 tx=0x7f2f54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86d7fff640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86c0003a30 0x7f86c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faf300aeb50 con 0x7faf38200590 2024-04-27T07:50:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72e05640 1 -- 172.21.15.22:0/1375161032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c0cf0e0 con 0x7ffb6c146180 2024-04-27T07:50:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7ffb72e05640 1 -- 172.21.15.22:0/1375161032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c0cf6a0 con 0x7ffb6c146180 2024-04-27T07:50:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 -- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86c0003a30 msgr2=0x7f86c0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86c0003a30 0x7f86c0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 -- 172.21.15.22:0/4286033239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c0004810 msgr2=0x7f86c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:50:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c0004810 0x7f86c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7f86dcda9640 1 -- 172.21.15.22:0/4286033239 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c0003530 con 0x7f86c00051d0 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7fce3d186640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3d186640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38073d00 0x7fce3809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3d186640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 0x7fce381579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3d186640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce380af110 con 0x7fce38072300 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3d186640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce380aef90 con 0x7fce38073d00 2024-04-27T07:50:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3d186640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce380af290 con 0x7fce380b4250 2024-04-27T07:50:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.500+0000 7faf2e7fc640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 0x7faf1c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faf30121430 con 0x7faf38200590 2024-04-27T07:50:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45162/0 (socket says 172.21.15.22:45162) 2024-04-27T07:50:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 -- 172.21.15.22:0/3114752064 learned_addr learned my addr 172.21.15.22:0/3114752064 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3c985640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 0x7fce381579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce377fe640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38073d00 0x7fce3809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 -- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 msgr2=0x7fce381579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 0x7fce381579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 -- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38073d00 msgr2=0x7fce3809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38073d00 0x7fce3809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 -- 172.21.15.22:0/3114752064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3814e9a0 con 0x7fce38072300 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce3c985640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 0x7fce381579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:50:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 Processor -- start 2024-04-27T07:50:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc9480bb020 con 0x7fc9540d0ce0 2024-04-27T07:50:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fc93b7fe640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 0x7fc94006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc948120c40 con 0x7fc9540d0ce0 2024-04-27T07:50:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 -- start start 2024-04-27T07:50:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7ffb72e05640 1 -- 172.21.15.22:0/1375161032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c04fea0 con 0x7ffb6c146180 2024-04-27T07:50:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7faf3d950640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 0x7faf1c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fc952ffd640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 0x7fc94006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 0x7f28340e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 0x7f283404aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28340df390 0x7f28340e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28340388b0 con 0x7f283404f5c0 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2834038730 con 0x7f28340df390 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f28428bf640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2834038a30 con 0x7f283404fa80 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f86dcda9640 1 -- 172.21.15.22:0/4286033239 reap_dead start 2024-04-27T07:50:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fce37fff640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce2800c3c0 tx=0x7fce2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 0x7f28340e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 0x7f28340e21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45166/0 (socket says 172.21.15.22:45166) 2024-04-27T07:50:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283bfff640 1 -- 172.21.15.22:0/2907855008 learned_addr learned my addr 172.21.15.22:0/2907855008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fca8e2ce640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca8806a7c0 tx=0x7fca88076e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7faf3d950640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 0x7faf1c06a8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 0x7f283404aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f86dcda9640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86c000b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009fc0 tx=0x7f86d001cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50866c8a10675716 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7fc952ffd640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 0x7fc94006a840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc94c003f50 tx=0x7fc94c0091b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f2840e5c640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28340df390 0x7f28340e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 -- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28340df390 msgr2=0x7f28340e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28340df390 0x7f28340e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 -- 172.21.15.22:0/2907855008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 msgr2=0x7f28340e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 0x7f28340e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 -- 172.21.15.22:0/2907855008 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283404aff0 con 0x7f283404fa80 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6400ce20 con 0x7ffb6c146180 2024-04-27T07:50:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64003ba0 con 0x7ffb6c146180 2024-04-27T07:50:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.504+0000 7f283b7fe640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 0x7f283404aab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f283400b520 tx=0x7f2830019220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28017070 con 0x7fce38072300 2024-04-27T07:50:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3809b0a0 con 0x7fce38072300 2024-04-27T07:50:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3809b660 con 0x7fce38072300 2024-04-27T07:50:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9480c2030 con 0x7fc9540d0ce0 2024-04-27T07:50:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3804fea0 con 0x7fce38072300 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88122070 con 0x7fca80007290 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca80106a30 con 0x7fca80007290 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80107a50 con 0x7fca80007290 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca800089e0 con 0x7fca80007290 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca88113620 con 0x7fca80007290 2024-04-27T07:50:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88118040 con 0x7fca80007290 2024-04-27T07:50:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28005cf0 con 0x7fce38072300 2024-04-27T07:50:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28021060 con 0x7fce38072300 2024-04-27T07:50:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb6400a850 con 0x7ffb6c146180 2024-04-27T07:50:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7ffb627fc640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 0x7ffb5806a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.508+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb64087370 con 0x7ffb6c146180 2024-04-27T07:50:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7ffb71e03640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 0x7ffb5806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce2801a070 con 0x7fce38072300 2024-04-27T07:50:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7fce34ff9640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 0x7fce2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7ffb71e03640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 0x7ffb5806a990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb68005890 tx=0x7ffb68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7fce377fe640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 0x7fce2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.512+0000 7fce377fe640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 0x7fce2406a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0af000 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300b0040 con 0x7f283404fa80 2024-04-27T07:50:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28300bb070 con 0x7f283404fa80 2024-04-27T07:50:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300ab440 con 0x7f283404fa80 2024-04-27T07:50:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f58018940 con 0x7f2f60073d00 2024-04-27T07:50:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb64020020 con 0x7ffb6c146180 2024-04-27T07:50:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283404dda0 con 0x7f283404fa80 2024-04-27T07:50:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28340cc1e0 con 0x7f283404fa80 2024-04-27T07:50:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28340cc7d0 con 0x7f283404fa80 2024-04-27T07:50:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fce280541e0 con 0x7fce38072300 2024-04-27T07:50:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fca88063230 con 0x7fca80007290 2024-04-27T07:50:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7fca7effd640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 0x7fca6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf300170d0 con 0x7faf38200590 2024-04-27T07:50:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fca8818b5c0 con 0x7fca80007290 2024-04-27T07:50:59.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.516+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0009c10 con 0x7f86c00051d0 2024-04-27T07:50:59.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7fca8dacd640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 0x7fca6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28300dbab0 con 0x7f283404fa80 2024-04-27T07:50:59.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f2838ff9640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 0x7f282006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2830120980 con 0x7f283404fa80 2024-04-27T07:50:59.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d00a5d30 con 0x7f86c00051d0 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86d6ffd640 1 -- 172.21.15.22:0/4286033239 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003be0 con 0x7f86c00051d0 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 msgr2=0x7f86c000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86c000b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009fc0 tx=0x7f86d001cce0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 shutdown_connections 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/4286033239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86c000b9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 >> 172.21.15.22:0/4286033239 conn(0x7f86d808bfc0 msgr2=0x7f86d80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 shutdown_connections 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- 172.21.15.22:0/4286033239 wait complete. 2024-04-27T07:50:59.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f283bfff640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 0x7f282006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f283bfff640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 0x7f282006a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7fca8dacd640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 0x7fca6c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca801a6e70 tx=0x7fca780079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 Processor -- start 2024-04-27T07:50:59.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.520+0000 7f86dd5aa640 1 -- start start 2024-04-27T07:50:59.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86d815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce28056140 con 0x7fce38072300 2024-04-27T07:50:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86d8156bc0 0x7f86d80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 0x7f86d80cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86d815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81608f0 con 0x7f86d8156600 2024-04-27T07:50:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86d77fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86d8156bc0 0x7f86d80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 0x7f86d80cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86d815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:54820/0 (socket says 172.21.15.22:54820) 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86d7fff640 1 -- 172.21.15.22:0/486192210 learned_addr learned my addr 172.21.15.22:0/486192210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86d80b5760 con 0x7f86c00051d0 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f86d80cfd00 con 0x7f86d8156bc0 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 -- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86d8156bc0 msgr2=0x7f86d80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86d8156bc0 0x7f86d80ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 -- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 msgr2=0x7f86d815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86d815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d815dd30 con 0x7f86d8156600 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.524+0000 7f86dcda9640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 0x7f86d80cd130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86d808ca20 tx=0x7f86d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0009c10 con 0x7f86d8156600 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d00bd070 con 0x7f86d8156600 2024-04-27T07:50:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003db0 con 0x7f86d8156600 2024-04-27T07:50:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca8812b020 con 0x7fca80007290 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d80c8c20 con 0x7f86d8156600 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d815d4e0 con 0x7f86d8156600 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d804fea0 con 0x7f86d8156600 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f86d00ae950 con 0x7f86d8156600 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.528+0000 7f86d4ff9640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 0x7f86c406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.532+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f86d0122090 con 0x7f86d8156600 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.532+0000 7f86d7fff640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 0x7f86c406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.532+0000 7f86d7fff640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 0x7f86c406a890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000fc0 tx=0x7f86cc007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:50:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.532+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2830016070 con 0x7f283404fa80 2024-04-27T07:50:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.540+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86d00c7020 con 0x7f86d8156600 2024-04-27T07:50:59.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.832+0000 7f2f64c12640 1 -- 172.21.15.22:0/3248386366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2f60150010 con 0x7f2f60073d00 2024-04-27T07:50:59.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.836+0000 7f2f47fff640 1 -- 172.21.15.22:0/3248386366 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f580f0cf0 con 0x7f2f60073d00 2024-04-27T07:50:59.841 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 msgr2=0x7f2f4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 0x7f2f4c06a840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f600b4240 tx=0x7f2f54009040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 msgr2=0x7f2f601481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f580b3040 tx=0x7f2f5801bac0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 shutdown_connections 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f4c068380 0x7f2f4c06a840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f60152090 0x7f2f601462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f600b0c10 0x7f2f601fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 --2- 172.21.15.22:0/3248386366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f60073d00 0x7f2f601481c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 >> 172.21.15.22:0/3248386366 conn(0x7f2f600b5890 msgr2=0x7f2f6015a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 shutdown_connections 2024-04-27T07:50:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7f2f45ffb640 1 -- 172.21.15.22:0/3248386366 wait complete. 2024-04-27T07:50:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7fc9596d6640 1 -- 172.21.15.22:0/4279148377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc954168410 con 0x7fc9540d0ce0 2024-04-27T07:50:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.840+0000 7fc93b7fe640 1 -- 172.21.15.22:0/4279148377 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc948120070 con 0x7fc9540d0ce0 2024-04-27T07:50:59.850 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7ffb72e05640 1 -- 172.21.15.22:0/1375161032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffb6c1455b0 con 0x7ffb6c146180 2024-04-27T07:50:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 msgr2=0x7fc94006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 0x7fc94006a840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc94c003f50 tx=0x7fc94c0091b0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 msgr2=0x7fc9540ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 0x7fc9540ad130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9480ab000 tx=0x7fc9480190f0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 shutdown_connections 2024-04-27T07:50:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc940068380 0x7fc94006a840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9540d0ce0 0x7fc9540ad130 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9540cbf90 0x7fc954151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7fc9397fa640 1 --2- 172.21.15.22:0/4279148377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc954073930 0x7fc9540999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.844+0000 7ffb627fc640 1 -- 172.21.15.22:0/1375161032 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb64057a20 con 0x7ffb6c146180 2024-04-27T07:50:59.853 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:50:59.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 >> 172.21.15.22:0/4279148377 conn(0x7fc9540d01c0 msgr2=0x7fc954153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 shutdown_connections 2024-04-27T07:50:59.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7fc9397fa640 1 -- 172.21.15.22:0/4279148377 wait complete. 2024-04-27T07:50:59.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 msgr2=0x7ffb5806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 0x7ffb5806a990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb68005890 tx=0x7ffb68009040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 msgr2=0x7ffb6c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400c710 tx=0x7ffb6400cbd0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 shutdown_connections 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb580684d0 0x7ffb5806a990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6c146180 0x7ffb6c0d1bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb6c0ae240 0x7ffb6c0d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 --2- 172.21.15.22:0/1375161032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb6c09aa30 0x7ffb6c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 >> 172.21.15.22:0/1375161032 conn(0x7ffb6c073450 msgr2=0x7ffb6c06f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 shutdown_connections 2024-04-27T07:50:59.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.848+0000 7ffb4ffff640 1 -- 172.21.15.22:0/1375161032 wait complete. 2024-04-27T07:50:59.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.856+0000 7faf3e952640 1 -- 172.21.15.22:0/442909689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faf380d0d00 con 0x7faf38200590 2024-04-27T07:50:59.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.852+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fce38166ce0 con 0x7fce38072300 2024-04-27T07:50:59.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.852+0000 7fce34ff9640 1 -- 172.21.15.22:0/3114752064 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce2805a440 con 0x7fce38072300 2024-04-27T07:50:59.862 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.856+0000 7faf2e7fc640 1 -- 172.21.15.22:0/442909689 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf300f1410 con 0x7faf38200590 2024-04-27T07:50:59.863 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 msgr2=0x7faf1c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 0x7faf1c06a8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 msgr2=0x7faf3815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 0x7faf3815e530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c540 tx=0x7faf24005840 comp rx=0 tx=0).stop 2024-04-27T07:50:59.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 msgr2=0x7fce2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 0x7fce2406a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0af000 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 msgr2=0x7fce38156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce2800c3c0 tx=0x7fce2800c880 comp rx=0 tx=0).stop 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 shutdown_connections 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce24068380 0x7fce2406a840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce380b4250 0x7fce381579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38073d00 0x7fce3809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 --2- 172.21.15.22:0/3114752064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce38072300 0x7fce38156a40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 >> 172.21.15.22:0/3114752064 conn(0x7fce380b5890 msgr2=0x7fce380afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 shutdown_connections 2024-04-27T07:50:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf1c068400 0x7faf1c06a8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf38200a10 0x7faf3815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf38200590 0x7faf3815e530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 --2- 172.21.15.22:0/442909689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf24004810 0x7faf381605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 >> 172.21.15.22:0/442909689 conn(0x7faf380d01c0 msgr2=0x7faf380ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 shutdown_connections 2024-04-27T07:50:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fce3d186640 1 -- 172.21.15.22:0/3114752064 wait complete. 2024-04-27T07:50:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 shutdown_connections 2024-04-27T07:50:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7faf1bfff640 1 -- 172.21.15.22:0/442909689 wait complete. 2024-04-27T07:50:59.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fca800053a0 con 0x7fca80007290 2024-04-27T07:50:59.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.860+0000 7fca7effd640 1 -- 172.21.15.22:0/729271303 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca88083600 con 0x7fca80007290 2024-04-27T07:50:59.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-04-27T07:50:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.868+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f28340053a0 con 0x7f283404fa80 2024-04-27T07:50:59.875 INFO:teuthology.orchestra.run.smithi022.stdout:60129542148 2024-04-27T07:50:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f86d80d38d0 con 0x7f86d8156600 2024-04-27T07:50:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f2838ff9640 1 -- 172.21.15.22:0/2907855008 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28300f0f20 con 0x7f283404fa80 2024-04-27T07:50:59.876 INFO:teuthology.orchestra.run.smithi022.stdout:60129542148 2024-04-27T07:50:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f86d4ff9640 1 -- 172.21.15.22:0/486192210 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86d00169f0 con 0x7f86d8156600 2024-04-27T07:50:59.877 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:50:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 msgr2=0x7f282006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 0x7f282006a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009040 comp rx=0 tx=0).stop 2024-04-27T07:50:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 msgr2=0x7fca6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 0x7fca6c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca801a6e70 tx=0x7fca780079c0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 msgr2=0x7f283404aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 msgr2=0x7fca80107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca8806a7c0 tx=0x7fca88076e70 comp rx=0 tx=0).stop 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 shutdown_connections 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca6c068380 0x7fca6c06a840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca80054610 0x7fca8004d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca80054230 0x7fca800f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7fca90532640 1 --2- 172.21.15.22:0/729271303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca80007290 0x7fca80107510 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca8806a7c0 tx=0x7fca88076e70 comp rx=0 tx=0).stop 2024-04-27T07:50:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 >> 172.21.15.22:0/729271303 conn(0x7fca80009fc0 msgr2=0x7fca8004e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.872+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 0x7f283404aab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f283400b520 tx=0x7f2830019220 comp rx=0 tx=0).stop 2024-04-27T07:50:59.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 shutdown_connections 2024-04-27T07:50:59.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2820068380 0x7f282006a840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28340df390 0x7f28340e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 shutdown_connections 2024-04-27T07:50:59.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f283404fa80 0x7f283404aab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7fca90532640 1 -- 172.21.15.22:0/729271303 wait complete. 2024-04-27T07:50:59.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 --2- 172.21.15.22:0/2907855008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f283404f5c0 0x7f28340e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 >> 172.21.15.22:0/2907855008 conn(0x7f2834009fc0 msgr2=0x7f28340535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 shutdown_connections 2024-04-27T07:50:59.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f28428bf640 1 -- 172.21.15.22:0/2907855008 wait complete. 2024-04-27T07:50:59.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 msgr2=0x7f86c406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 0x7f86c406a890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000fc0 tx=0x7f86cc007cc0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 msgr2=0x7f86d80cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 0x7f86d80cd130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86d808ca20 tx=0x7f86d0009750 comp rx=0 tx=0).stop 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 shutdown_connections 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f86c40683d0 0x7f86c406a890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86d8156600 0x7f86d80cd130 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86d8156bc0 0x7f86d80ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 --2- 172.21.15.22:0/486192210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86c00051d0 0x7f86d815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:50:59.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 >> 172.21.15.22:0/486192210 conn(0x7f86d808bfc0 msgr2=0x7f86d80d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:50:59.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 shutdown_connections 2024-04-27T07:50:59.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:50:59.876+0000 7f86dd5aa640 1 -- 172.21.15.22:0/486192210 wait complete. 2024-04-27T07:50:59.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-04-27T07:50:59.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.7 2024-04-27T07:50:59.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-04-27T07:50:59.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-04-27T07:50:59.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-04-27T07:50:59.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.5 2024-04-27T07:50:59.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-04-27T07:51:00.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T07:51:00.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T07:51:00.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-27T07:51:00.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T07:51:00.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-27T07:51:00.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T07:51:00.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-27T07:51:00.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-27T07:51:01.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.052+0000 7f3686d86640 1 Processor -- start 2024-04-27T07:51:01.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.052+0000 7f3686d86640 1 -- start start 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3686d86640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f368015dba0 0x7f36800cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3686d86640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3686d86640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36800cb2c0 con 0x7f36800d0ce0 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3686d86640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f36800cbbf0 con 0x7f368015dba0 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3686d86640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f36800cb5a0 con 0x7f36800ce2d0 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f368015dba0 0x7f36800cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f368015dba0 0x7f36800cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40658/0 (socket says 172.21.15.22:40658) 2024-04-27T07:51:01.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 -- 172.21.15.22:0/46628436 learned_addr learned my addr 172.21.15.22:0/46628436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f3685323640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 -- 172.21.15.22:0/46628436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 msgr2=0x7f36800ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 -- 172.21.15.22:0/46628436 >> v1:172.21.15.183:6789/0 conn(0x7f36800d0ce0 legacy=0x7f36800cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.056+0000 7f367ffff640 1 -- 172.21.15.22:0/46628436 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36800cb880 con 0x7f368015dba0 2024-04-27T07:51:01.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f367ffff640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f36800cdd90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3678009e50 tx=0x7f367801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c0babf97469ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f3685323640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:01.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f367f7fe640 1 -- 172.21.15.22:0/46628436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36780a5850 con 0x7f368015dba0 2024-04-27T07:51:01.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f367f7fe640 1 -- 172.21.15.22:0/46628436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36780a59f0 con 0x7f368015dba0 2024-04-27T07:51:01.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f367f7fe640 1 -- 172.21.15.22:0/46628436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36780a5dc0 con 0x7f368015dba0 2024-04-27T07:51:01.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f3686d86640 1 -- 172.21.15.22:0/46628436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 msgr2=0x7f36800cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.060+0000 7f3686d86640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f36800cdd90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3678009e50 tx=0x7f367801ca70 comp rx=0 tx=0).stop 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 -- 172.21.15.22:0/46628436 shutdown_connections 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 --2- 172.21.15.22:0/46628436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f36800cdd90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 -- 172.21.15.22:0/46628436 >> 172.21.15.22:0/46628436 conn(0x7f36800d01c0 msgr2=0x7f36800d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 -- 172.21.15.22:0/46628436 shutdown_connections 2024-04-27T07:51:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 -- 172.21.15.22:0/46628436 wait complete. 2024-04-27T07:51:01.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.072+0000 7f3686d86640 1 Processor -- start 2024-04-27T07:51:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 -- start start 2024-04-27T07:51:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36800d0ce0 0x7f368009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f368014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36800addb0 con 0x7f368015dba0 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36800adc30 con 0x7f36800d0ce0 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3686d86640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f36800adf30 con 0x7f36800ce2d0 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3685323640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f368014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55296/0 (socket says 172.21.15.22:55296) 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 -- 172.21.15.22:0/1399354389 learned_addr learned my addr 172.21.15.22:0/1399354389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3685323640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f368014d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40664/0 (socket says 172.21.15.22:40664) 2024-04-27T07:51:01.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f367ffff640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36800d0ce0 0x7f368009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 -- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36800d0ce0 msgr2=0x7f368009db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36800d0ce0 0x7f368009db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 -- 172.21.15.22:0/1399354389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 msgr2=0x7f368014d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f368014d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 -- 172.21.15.22:0/1399354389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368014a860 con 0x7f36800ce2d0 2024-04-27T07:51:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.076+0000 7f3684b22640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f367400c500 tx=0x7f367400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674016020 con 0x7f36800ce2d0 2024-04-27T07:51:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f3686d86640 1 -- 172.21.15.22:0/1399354389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368014ab40 con 0x7f36800ce2d0 2024-04-27T07:51:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f3686d86640 1 -- 172.21.15.22:0/1399354389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f368014b100 con 0x7f36800ce2d0 2024-04-27T07:51:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674005220 con 0x7f36800ce2d0 2024-04-27T07:51:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007030 con 0x7f36800ce2d0 2024-04-27T07:51:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.080+0000 7f3686d86640 1 -- 172.21.15.22:0/1399354389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c0052a0 con 0x7f36800ce2d0 2024-04-27T07:51:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.084+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3674036a60 con 0x7f36800ce2d0 2024-04-27T07:51:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.084+0000 7f367d7fa640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 0x7f366406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.084+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f36740864a0 con 0x7f36800ce2d0 2024-04-27T07:51:01.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.084+0000 7f367ffff640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 0x7f366406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.084+0000 7f367ffff640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 0x7f366406a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36780ae000 tx=0x7f36780a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.096+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3674020020 con 0x7f36800ce2d0 2024-04-27T07:51:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 Processor -- start 2024-04-27T07:51:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 -- start start 2024-04-27T07:51:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 Processor -- start 2024-04-27T07:51:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 -- start start 2024-04-27T07:51:01.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1b880b3af0 0x7f1b880cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b880cc1f0 con 0x7f1b880b0800 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1b880cce00 con 0x7f1b880b3af0 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8f038640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1b880cc4d0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55304/0 (socket says 172.21.15.22:55304) 2024-04-27T07:51:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 -- 172.21.15.22:0/4225993523 learned_addr learned my addr 172.21.15.22:0/4225993523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7fb2c4285640 1 Processor -- start 2024-04-27T07:51:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8d835640 1 --2- 172.21.15.22:0/4225993523 >> v2:172.21.15.183:3300/0 conn(0x7f1b880b3af0 0x7f1b880cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7fb2c4285640 1 -- start start 2024-04-27T07:51:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f013415ea60 0x7f01340cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 0x7f0134070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013406bf20 con 0x7f0134070ca0 2024-04-27T07:51:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f013406c200 con 0x7f013415ea60 2024-04-27T07:51:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f013b4eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f013406c4e0 con 0x7f01340cd620 2024-04-27T07:51:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 -- 172.21.15.22:0/4225993523 >> v2:172.21.15.183:3300/0 conn(0x7f1b880b3af0 msgr2=0x7f1b880cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 --2- 172.21.15.22:0/4225993523 >> v2:172.21.15.183:3300/0 conn(0x7f1b880b3af0 0x7f1b880cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 -- 172.21.15.22:0/4225993523 >> v1:172.21.15.22:6789/0 conn(0x7f1b880b0800 legacy=0x7f1b8806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f1b8e036640 1 -- 172.21.15.22:0/4225993523 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b880cc7b0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f0139287640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f013415ea60 0x7f01340cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f0138a86640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 0x7f0134070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f0139287640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f013415ea60 0x7f01340cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40678/0 (socket says 172.21.15.22:40678) 2024-04-27T07:51:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f0138a86640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 0x7f0134070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55320/0 (socket says 172.21.15.22:55320) 2024-04-27T07:51:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.104+0000 7f0139287640 1 -- 172.21.15.22:0/958743409 learned_addr learned my addr 172.21.15.22:0/958743409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f0139287640 1 -- 172.21.15.22:0/958743409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 msgr2=0x7f0134070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f0139287640 1 --2- 172.21.15.22:0/958743409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 0x7f0134070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f0139287640 1 -- 172.21.15.22:0/958743409 >> v1:172.21.15.183:6789/0 conn(0x7f0134070ca0 legacy=0x7f013406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f0139287640 1 -- 172.21.15.22:0/958743409 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013406c7c0 con 0x7f013415ea60 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 Processor -- start 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 -- start start 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9900d0c40 0x7fc99006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900cfbe0 con 0x7fc9900c4460 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc990070a90 con 0x7fc9900702e0 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc9975de640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc9900d07b0 con 0x7fc9900d0c40 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9900d0c40 0x7fc99006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc9900d0c40 0x7fc99006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40710/0 (socket says 172.21.15.22:40710) 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 -- 172.21.15.22:0/127622095 learned_addr learned my addr 172.21.15.22:0/127622095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c4285640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2bc151cf0 0x7fb2bc0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c4285640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb2bc0b5740 0x7fb2bc0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c4285640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2bc1520d0 con 0x7fb2bc0cea10 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c4285640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc0b3590 con 0x7fb2bc151cf0 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c4285640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc0b3870 con 0x7fb2bc0b5740 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2822640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb2bc0cea10 0x7fb2bc15e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41190/0 (socket says 172.21.15.22:41190) 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2822640 1 -- 172.21.15.22:0/733435592 learned_addr learned my addr 172.21.15.22:0/733435592 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2021640 1 --2- 172.21.15.22:0/733435592 >> v2:172.21.15.22:3300/0 conn(0x7fb2bc151cf0 0x7fb2bc0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c1820640 1 --2- 172.21.15.22:0/733435592 >> v2:172.21.15.183:3300/0 conn(0x7fb2bc0b5740 0x7fb2bc0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2021640 1 -- 172.21.15.22:0/733435592 >> v2:172.21.15.183:3300/0 conn(0x7fb2bc0b5740 msgr2=0x7fb2bc0728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2021640 1 --2- 172.21.15.22:0/733435592 >> v2:172.21.15.183:3300/0 conn(0x7fb2bc0b5740 0x7fb2bc0728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2021640 1 -- 172.21.15.22:0/733435592 >> v1:172.21.15.22:6789/0 conn(0x7fb2bc0cea10 legacy=0x7fb2bc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fb2c2021640 1 -- 172.21.15.22:0/733435592 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc0b3b50 con 0x7fb2bc151cf0 2024-04-27T07:51:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 -- 172.21.15.22:0/127622095 >> v1:172.21.15.183:6789/0 conn(0x7fc9900702e0 legacy=0x7fc9900ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 -- 172.21.15.22:0/127622095 >> v1:172.21.15.22:6789/0 conn(0x7fc9900c4460 legacy=0x7fc9900c4840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:51:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f1b8e036640 1 --2- 172.21.15.22:0/4225993523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009870 tx=0x7f1b8401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace302a0a3f9a26c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f1b8d034640 1 -- 172.21.15.22:0/4225993523 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b840b5070 con 0x7f1b8815b5f0 2024-04-27T07:51:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f0139287640 1 --2- 172.21.15.22:0/958743409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f013415ea60 0x7f01340cbae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0124009870 tx=0x7f012401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cd40d05581d2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f1b8d034640 1 -- 172.21.15.22:0/4225993523 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b840a8a50 con 0x7f1b8815b5f0 2024-04-27T07:51:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7f1b8d034640 1 -- 172.21.15.22:0/4225993523 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b840a8e20 con 0x7f1b8815b5f0 2024-04-27T07:51:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- 172.21.15.22:0/4225993523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 msgr2=0x7f1b880cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- 172.21.15.22:0/4225993523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009870 tx=0x7f1b8401ed50 comp rx=0 tx=0).stop 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- 172.21.15.22:0/4225993523 shutdown_connections 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- 172.21.15.22:0/4225993523 >> v2:172.21.15.183:3300/0 conn(0x7f1b880b3af0 0x7f1b880cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- 172.21.15.22:0/4225993523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b880cd290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- 172.21.15.22:0/4225993523 >> 172.21.15.22:0/4225993523 conn(0x7f1b880b2fd0 msgr2=0x7f1b880b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- 172.21.15.22:0/4225993523 shutdown_connections 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- 172.21.15.22:0/4225993523 wait complete. 2024-04-27T07:51:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 Processor -- start 2024-04-27T07:51:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f012bfff640 1 -- 172.21.15.22:0/958743409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240ab650 con 0x7f013415ea60 2024-04-27T07:51:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.108+0000 7fc994b79640 1 -- 172.21.15.22:0/127622095 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900cfe80 con 0x7fc9900d0c40 2024-04-27T07:51:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f012bfff640 1 -- 172.21.15.22:0/958743409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240b0070 con 0x7f013415ea60 2024-04-27T07:51:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f012bfff640 1 -- 172.21.15.22:0/958743409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240aba10 con 0x7f013415ea60 2024-04-27T07:51:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c2021640 1 --2- 172.21.15.22:0/733435592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc0b5200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc158d70 tx=0x7fb2b801d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8725ddf1833b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c101f640 1 -- 172.21.15.22:0/733435592 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80aa680 con 0x7fb2bc151cf0 2024-04-27T07:51:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c101f640 1 -- 172.21.15.22:0/733435592 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b80aa820 con 0x7fb2bc151cf0 2024-04-27T07:51:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c101f640 1 -- 172.21.15.22:0/733435592 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80aac30 con 0x7fb2bc151cf0 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 -- start start 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- 172.21.15.22:0/958743409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f013415ea60 msgr2=0x7f01340cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 --2- 172.21.15.22:0/958743409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f013415ea60 0x7f01340cbae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0124009870 tx=0x7f012401c580 comp rx=0 tx=0).stop 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- 172.21.15.22:0/958743409 shutdown_connections 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 --2- 172.21.15.22:0/958743409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01340cd620 0x7f0134070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 --2- 172.21.15.22:0/958743409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f013415ea60 0x7f01340cbae0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- 172.21.15.22:0/958743409 >> 172.21.15.22:0/958743409 conn(0x7f01340b3020 msgr2=0x7f01340b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc994b79640 1 --2- 172.21.15.22:0/127622095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900d0c40 0x7fc99006fda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3635de733ca6736 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- 172.21.15.22:0/958743409 shutdown_connections 2024-04-27T07:51:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 Processor -- start 2024-04-27T07:51:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- start start 2024-04-27T07:51:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc987fff640 1 -- 172.21.15.22:0/127622095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0aa650 con 0x7fc9900d0c40 2024-04-27T07:51:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc987fff640 1 -- 172.21.15.22:0/127622095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c0af070 con 0x7fc9900d0c40 2024-04-27T07:51:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc987fff640 1 -- 172.21.15.22:0/127622095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0aaa10 con 0x7fc9900d0c40 2024-04-27T07:51:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 -- 172.21.15.22:0/733435592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc151cf0 msgr2=0x7fb2bc0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 --2- 172.21.15.22:0/733435592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc0b5200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc158d70 tx=0x7fb2b801d610 comp rx=0 tx=0).stop 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 -- 172.21.15.22:0/733435592 shutdown_connections 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 --2- 172.21.15.22:0/733435592 >> v2:172.21.15.183:3300/0 conn(0x7fb2bc0b5740 0x7fb2bc0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 --2- 172.21.15.22:0/733435592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc0b5200 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fb2c4285640 1 -- 172.21.15.22:0/733435592 >> 172.21.15.22:0/733435592 conn(0x7fb2bc067f10 msgr2=0x7fb2bc06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 -- 172.21.15.22:0/127622095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900d0c40 msgr2=0x7fc99006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 --2- 172.21.15.22:0/127622095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900d0c40 0x7fc99006fda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c01c4f0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 -- 172.21.15.22:0/127622095 shutdown_connections 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 --2- 172.21.15.22:0/127622095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900d0c40 0x7fc99006fda0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 -- 172.21.15.22:0/127622095 >> 172.21.15.22:0/127622095 conn(0x7fc9900b3020 msgr2=0x7fc9900b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b880b0800 0x7f1b8809eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b880b3af0 0x7f1b880b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8815b9d0 con 0x7f1b880b3af0 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880cbef0 con 0x7f1b880b0800 2024-04-27T07:51:01.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8f038640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1b880cd7d0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 -- 172.21.15.22:0/127622095 shutdown_connections 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7fc9975de640 1 -- 172.21.15.22:0/127622095 wait complete. 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffb8815cc70 0x7ffb88158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb880b41d0 0x7ffb880b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb88162200 con 0x7ffb880b4630 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8806fcb0 con 0x7ffb8815cc70 2024-04-27T07:51:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8759e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8806f0a0 con 0x7ffb880b41d0 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffb8815cc70 0x7ffb88158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffb8815cc70 0x7ffb88158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40718/0 (socket says 172.21.15.22:40718) 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 -- 172.21.15.22:0/2784561196 learned_addr learned my addr 172.21.15.22:0/2784561196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb85d9b640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb880b41d0 0x7ffb880b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 -- 172.21.15.22:0/2784561196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb880b41d0 msgr2=0x7ffb880b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb880b41d0 0x7ffb880b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 -- 172.21.15.22:0/2784561196 >> v1:172.21.15.22:6789/0 conn(0x7ffb880b4630 legacy=0x7ffb88070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7ffb8659c640 1 -- 172.21.15.22:0/2784561196 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8806f380 con 0x7ffb8815cc70 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- 172.21.15.22:0/958743409 wait complete. 2024-04-27T07:51:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 Processor -- start 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f1b8e837640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55350/0 (socket says 172.21.15.22:55350) 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 -- 172.21.15.22:0/3163833329 learned_addr learned my addr 172.21.15.22:0/3163833329 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e036640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b880b0800 0x7f1b8809eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 -- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b880b0800 msgr2=0x7f1b8809eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b880b0800 0x7f1b8809eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 -- 172.21.15.22:0/3163833329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b880b3af0 msgr2=0x7f1b880b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b880b3af0 0x7f1b880b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 -- 172.21.15.22:0/3163833329 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8809f410 con 0x7f1b8815b5f0 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.112+0000 7f013b4eb640 1 -- start start 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 0x7f0134205230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01340cd620 0x7f01340b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f013406bbd0 con 0x7f0134070ca0 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01340701a0 con 0x7f01340cd620 2024-04-27T07:51:01.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f013b4eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f013415ee40 con 0x7f013415ea60 2024-04-27T07:51:01.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 Processor -- start 2024-04-27T07:51:01.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 -- start start 2024-04-27T07:51:01.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fb2c4285640 1 -- 172.21.15.22:0/733435592 shutdown_connections 2024-04-27T07:51:01.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fb2c4285640 1 -- 172.21.15.22:0/733435592 wait complete. 2024-04-27T07:51:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8e837640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c710 tx=0x7f1b8000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139a88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0138a86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01340cd620 0x7f01340b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139a88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55358/0 (socket says 172.21.15.22:55358) 2024-04-27T07:51:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139a88640 1 -- 172.21.15.22:0/4040136169 learned_addr learned my addr 172.21.15.22:0/4040136169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 0x7f0134205230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80016020 con 0x7f1b8815b5f0 2024-04-27T07:51:01.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 -- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 msgr2=0x7f01340b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8659c640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8815cc70 0x7ffb88158ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb780098d0 tx=0x7ffb7802d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8090c39141fad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 -- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01340cd620 msgr2=0x7f01340b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01340cd620 0x7f01340b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 -- 172.21.15.22:0/4040136169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340b0c90 con 0x7f0134070ca0 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139a88640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:01.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9900c4460 0x7fc990154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9900d0c40 0x7fc990152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc990163300 con 0x7fc9900702e0 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8559a640 1 -- 172.21.15.22:0/2784561196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb780bc660 con 0x7ffb8815cc70 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc990163180 con 0x7fc9900d0c40 2024-04-27T07:51:01.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc9975de640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc990170a60 con 0x7fc9900c4460 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8559a640 1 -- 172.21.15.22:0/2784561196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb780c1070 con 0x7ffb8815cc70 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8559a640 1 -- 172.21.15.22:0/2784561196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb780bca20 con 0x7ffb8815cc70 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40748/0 (socket says 172.21.15.22:40748) 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 -- 172.21.15.22:0/1359798742 learned_addr learned my addr 172.21.15.22:0/1359798742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc994b79640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9900c4460 0x7fc990154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0139287640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 0x7f0134205230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01240ac000 tx=0x7f012401adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc995b7b640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9900d0c40 0x7fc990152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 -- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9900c4460 msgr2=0x7fc990154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9900c4460 0x7fc990154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 -- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9900d0c40 msgr2=0x7fc990152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9900d0c40 0x7fc990152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fc99537a640 1 -- 172.21.15.22:0/1359798742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900cbac0 con 0x7fc9900702e0 2024-04-27T07:51:01.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8809b7c0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8809bd80 con 0x7f1b8815b5f0 2024-04-27T07:51:01.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80006d60 con 0x7f1b8815b5f0 2024-04-27T07:51:01.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80003ba0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7fb2c4285640 1 Processor -- start 2024-04-27T07:51:01.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240be5f0 con 0x7f0134070ca0 2024-04-27T07:51:01.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240be790 con 0x7f0134070ca0 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 -- 172.21.15.22:0/2784561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8815cc70 msgr2=0x7ffb88158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8815cc70 0x7ffb88158ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb780098d0 tx=0x7ffb7802d540 comp rx=0 tx=0).stop 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 -- 172.21.15.22:0/2784561196 shutdown_connections 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb880b41d0 0x7ffb880b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 --2- 172.21.15.22:0/2784561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8815cc70 0x7ffb88158ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 -- 172.21.15.22:0/2784561196 >> 172.21.15.22:0/2784561196 conn(0x7ffb880b66f0 msgr2=0x7ffb880b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 -- 172.21.15.22:0/2784561196 shutdown_connections 2024-04-27T07:51:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240beb80 con 0x7f0134070ca0 2024-04-27T07:51:01.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fc99537a640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c450 tx=0x7fc98000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01340cad00 con 0x7f0134070ca0 2024-04-27T07:51:01.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01340cb2c0 con 0x7f0134070ca0 2024-04-27T07:51:01.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f01177fe640 1 -- 172.21.15.22:0/4040136169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01100052a0 con 0x7f0134070ca0 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980012830 con 0x7fc9900702e0 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9900cbd60 con 0x7fc9900702e0 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 -- start start 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 0x7fb2bc204140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2bc0cea10 0x7fb2bc1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc15f0b0 con 0x7fb2bc0b5740 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc15ef30 con 0x7fb2bc151cf0 2024-04-27T07:51:01.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c4285640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc15f230 con 0x7fb2bc0cea10 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:55466/0 (socket says 172.21.15.22:55466) 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 -- 172.21.15.22:0/2628499435 learned_addr learned my addr 172.21.15.22:0/2628499435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.116+0000 7ffb8759e640 1 -- 172.21.15.22:0/2784561196 wait complete. 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7ffb8759e640 1 Processor -- start 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2021640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 0x7fb2bc204140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c1820640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2bc0cea10 0x7fb2bc1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1b8000a850 con 0x7f1b8815b5f0 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 -- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2bc0cea10 msgr2=0x7fb2bc1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2bc0cea10 0x7fb2bc1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 -- 172.21.15.22:0/2628499435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 msgr2=0x7fb2bc204140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 0x7fb2bc204140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 -- 172.21.15.22:0/2628499435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1531b0 con 0x7fb2bc151cf0 2024-04-27T07:51:01.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9900cc320 con 0x7fc9900702e0 2024-04-27T07:51:01.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2021640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 0x7fb2bc204140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:01.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990069340 con 0x7fc9900702e0 2024-04-27T07:51:01.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f1b7effd640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 0x7f1b6c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b80086590 con 0x7f1b8815b5f0 2024-04-27T07:51:01.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f1b8e036640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 0x7f1b6c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b700052a0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7ffb8759e640 1 -- start start 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7ffb8759e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb880b41d0 0x7ffb88164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb8759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 0x7ffb88152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb8759e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb8759e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88159410 con 0x7ffb880b4630 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb8759e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8815c950 con 0x7ffb880b41d0 2024-04-27T07:51:01.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb8759e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffb88070c60 con 0x7ffb8815cc70 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2c2822640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400caa0 tx=0x7fb2b400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4017070 con 0x7fb2bc151cf0 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b400ac00 con 0x7fb2bc151cf0 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.120+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4012620 con 0x7fb2bc151cf0 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc15fbe0 con 0x7fb2bc151cf0 2024-04-27T07:51:01.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 Processor -- start 2024-04-27T07:51:01.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc160480 con 0x7fb2bc151cf0 2024-04-27T07:51:01.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980017070 con 0x7fc9900702e0 2024-04-27T07:51:01.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980012bd0 con 0x7fc9900702e0 2024-04-27T07:51:01.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 -- start start 2024-04-27T07:51:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55378/0 (socket says 172.21.15.22:55378) 2024-04-27T07:51:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 -- 172.21.15.22:0/3439071473 learned_addr learned my addr 172.21.15.22:0/3439071473 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc15fdb0 con 0x7fb2bc151cf0 2024-04-27T07:51:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01240c9430 con 0x7f0134070ca0 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f0129ffb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 0x7f011806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efdac162da0 0x7efdac06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efdac0ce260 0x7efdac0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdac06e630 con 0x7efdac06f560 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efdac163180 con 0x7efdac162da0 2024-04-27T07:51:01.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb44e8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7efdac06f1b0 con 0x7efdac0ce260 2024-04-27T07:51:01.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f01241212b0 con 0x7f0134070ca0 2024-04-27T07:51:01.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb85d9b640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 0x7ffb88152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 -- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb880b41d0 msgr2=0x7ffb88164f40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-27T07:51:01.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb880b41d0 0x7ffb88164f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 -- 172.21.15.22:0/3439071473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 msgr2=0x7ffb88152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f0138a86640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 0x7f011806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb2a85640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7efdac06f560 0x7efdac06f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41218/0 (socket says 172.21.15.22:41218) 2024-04-27T07:51:01.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb2a85640 1 -- 172.21.15.22:0/3359545710 learned_addr learned my addr 172.21.15.22:0/3359545710 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:01.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb2284640 1 --2- 172.21.15.22:0/3359545710 >> v2:172.21.15.22:3300/0 conn(0x7efdac162da0 0x7efdac06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 0x7ffb88152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 -- 172.21.15.22:0/3439071473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb88150860 con 0x7ffb8815cc70 2024-04-27T07:51:01.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb85d9b640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 0x7ffb88152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:01.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb1a83640 1 --2- 172.21.15.22:0/3359545710 >> v2:172.21.15.183:3300/0 conn(0x7efdac0ce260 0x7efdac0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7ffb86d9d640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c760 tx=0x7ffb8000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb1a83640 1 -- 172.21.15.22:0/3359545710 >> v2:172.21.15.22:3300/0 conn(0x7efdac162da0 msgr2=0x7efdac06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb1a83640 1 --2- 172.21.15.22:0/3359545710 >> v2:172.21.15.22:3300/0 conn(0x7efdac162da0 0x7efdac06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb1a83640 1 -- 172.21.15.22:0/3359545710 >> v1:172.21.15.22:6789/0 conn(0x7efdac06f560 legacy=0x7efdac06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7efdb1a83640 1 -- 172.21.15.22:0/3359545710 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac06e910 con 0x7efdac0ce260 2024-04-27T07:51:01.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f0138a86640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 0x7f011806a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f01340cbc00 tx=0x7f0130009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb2b401a070 con 0x7fb2bc151cf0 2024-04-27T07:51:01.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.124+0000 7f2948c1a640 1 Processor -- start 2024-04-27T07:51:01.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7fb2b2ffd640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 0x7fb2a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b4086b60 con 0x7fb2bc151cf0 2024-04-27T07:51:01.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0124016170 con 0x7f0134070ca0 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 -- start start 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29440d0dc0 0x7f29440d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f29440b7410 0x7f29440cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29440b5600 con 0x7f29440d2ba0 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f29440d13a0 con 0x7f29440d0dc0 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2948c1a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f29440d1680 con 0x7f29440b7410 2024-04-27T07:51:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f1b8e036640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 0x7f1b6c06a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b840b2000 tx=0x7f1b840aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29440d0dc0 0x7f29440d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29440d0dc0 0x7f29440d5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40784/0 (socket says 172.21.15.22:40784) 2024-04-27T07:51:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 -- 172.21.15.22:0/1892459919 learned_addr learned my addr 172.21.15.22:0/1892459919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f29437fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f29440d2ba0 0x7f29440d2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41232/0 (socket says 172.21.15.22:41232) 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f2943fff640 1 --2- 172.21.15.22:0/1892459919 >> v2:172.21.15.183:3300/0 conn(0x7f29440b7410 0x7f29440cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 -- 172.21.15.22:0/1892459919 >> v2:172.21.15.183:3300/0 conn(0x7f29440b7410 msgr2=0x7f29440cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 --2- 172.21.15.22:0/1892459919 >> v2:172.21.15.183:3300/0 conn(0x7f29440b7410 0x7f29440cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 -- 172.21.15.22:0/1892459919 >> v1:172.21.15.22:6789/0 conn(0x7f29440d2ba0 legacy=0x7f29440d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7f293bfff640 1 -- 172.21.15.22:0/1892459919 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29440d1960 con 0x7f29440d0dc0 2024-04-27T07:51:01.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7fb2c2021640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 0x7fb2a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7efdb1a83640 1 --2- 172.21.15.22:0/3359545710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac0b3e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efdac0734b0 tx=0x7efd9c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d40953e4110860 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7fb2c2021640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 0x7fb2a006a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80af000 tx=0x7fb2b80a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.128+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc98001c460 con 0x7fc9900702e0 2024-04-27T07:51:01.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7fc985ffb640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 0x7fc97406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc980086db0 con 0x7fc9900702e0 2024-04-27T07:51:01.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7f293bfff640 1 --2- 172.21.15.22:0/1892459919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440d0dc0 0x7f29440d5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f294408d930 tx=0x7f292c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1f7251e499cdcf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7fc994b79640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 0x7fc97406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7fc994b79640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 0x7fc97406a840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0ae000 tx=0x7fc98c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc980057390 con 0x7fc9900702e0 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7f2942ffd640 1 -- 172.21.15.22:0/1892459919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0b1070 con 0x7f29440d0dc0 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7f2942ffd640 1 -- 172.21.15.22:0/1892459919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c0ac600 con 0x7f29440d0dc0 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.132+0000 7f2942ffd640 1 -- 172.21.15.22:0/1892459919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0ac9d0 con 0x7f29440d0dc0 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- 172.21.15.22:0/1892459919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440d0dc0 msgr2=0x7f29440d5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 --2- 172.21.15.22:0/1892459919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440d0dc0 0x7f29440d5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f294408d930 tx=0x7f292c01c490 comp rx=0 tx=0).stop 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- 172.21.15.22:0/1892459919 shutdown_connections 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 --2- 172.21.15.22:0/1892459919 >> v2:172.21.15.183:3300/0 conn(0x7f29440b7410 0x7f29440cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 --2- 172.21.15.22:0/1892459919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440d0dc0 0x7f29440d5020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- 172.21.15.22:0/1892459919 >> 172.21.15.22:0/1892459919 conn(0x7f294408bfc0 msgr2=0x7f29440d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- 172.21.15.22:0/1892459919 shutdown_connections 2024-04-27T07:51:01.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b80020020 con 0x7f1b8815b5f0 2024-04-27T07:51:01.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- 172.21.15.22:0/1892459919 wait complete. 2024-04-27T07:51:01.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 Processor -- start 2024-04-27T07:51:01.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7f2948c1a640 1 -- start start 2024-04-27T07:51:01.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80003ba0 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb80003d60 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb88150b40 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000aa90 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.136+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb88151100 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8804fea0 con 0x7ffb8815cc70 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2b4021020 con 0x7fb2bc151cf0 2024-04-27T07:51:01.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7efdb1282640 1 -- 172.21.15.22:0/3359545710 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0b0070 con 0x7efdac0ce260 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7efdb1282640 1 -- 172.21.15.22:0/3359545710 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c0ab600 con 0x7efdac0ce260 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7efdb1282640 1 -- 172.21.15.22:0/3359545710 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0ab9d0 con 0x7efdac0ce260 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7efdb44e8640 1 -- 172.21.15.22:0/3359545710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 msgr2=0x7efdac0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7efdb44e8640 1 --2- 172.21.15.22:0/3359545710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac0b3e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efdac0734b0 tx=0x7efd9c01c480 comp rx=0 tx=0).stop 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29440d0dc0 0x7f2944203fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29440d2ba0 0x7f294409f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f294416b8f0 con 0x7f29440b7410 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f294416b770 con 0x7f29440d0dc0 2024-04-27T07:51:01.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f2948c1a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f294416ba70 con 0x7f29440d2ba0 2024-04-27T07:51:01.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb8000ac30 con 0x7ffb8815cc70 2024-04-27T07:51:01.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7ffb76ffd640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 0x7ffb6806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb80087310 con 0x7ffb8815cc70 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40788/0 (socket says 172.21.15.22:40788) 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 -- 172.21.15.22:0/3324184919 learned_addr learned my addr 172.21.15.22:0/3324184919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 -- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29440d2ba0 msgr2=0x7f294409f660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29440d2ba0 0x7f294409f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 -- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29440d0dc0 msgr2=0x7f2944203fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29440d0dc0 0x7f2944203fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.140+0000 7f29437fe640 1 -- 172.21.15.22:0/3324184919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29440c7f50 con 0x7f29440b7410 2024-04-27T07:51:01.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7ffb8659c640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 0x7ffb6806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f29437fe640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f293400c450 tx=0x7f293400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934018070 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2934007480 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29440c8230 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29440c87f0 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29340047f0 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.144+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f293401b070 con 0x7f29440b7410 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- 172.21.15.22:0/3359545710 shutdown_connections 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 --2- 172.21.15.22:0/3359545710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac0b3e70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 --2- 172.21.15.22:0/3359545710 >> v2:172.21.15.22:3300/0 conn(0x7efdac162da0 0x7efdac06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- 172.21.15.22:0/3359545710 >> 172.21.15.22:0/3359545710 conn(0x7efdac067f10 msgr2=0x7efdac06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7f2940ff9640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 0x7f292406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- 172.21.15.22:0/3359545710 shutdown_connections 2024-04-27T07:51:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2934087dd0 con 0x7f29440b7410 2024-04-27T07:51:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7f293bfff640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 0x7f292406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7ffb8659c640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 0x7ffb6806a890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb780c0000 tx=0x7ffb780b8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- 172.21.15.22:0/3359545710 wait complete. 2024-04-27T07:51:01.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7f293bfff640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 0x7f292406a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f292c0ae000 tx=0x7f292c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 Processor -- start 2024-04-27T07:51:01.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29280052a0 con 0x7f29440b7410 2024-04-27T07:51:01.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- start start 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac15be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdac162da0 0x7efdac0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdac0b5700 con 0x7efdac06f560 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdac0b5580 con 0x7efdac0ce260 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.148+0000 7efdb44e8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efdac0b5880 con 0x7efdac162da0 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40796/0 (socket says 172.21.15.22:40796) 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 -- 172.21.15.22:0/1038088793 learned_addr learned my addr 172.21.15.22:0/1038088793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb1a83640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac15be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2a85640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdac162da0 0x7efdac0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 -- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdac162da0 msgr2=0x7efdac0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdac162da0 0x7efdac0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 -- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 msgr2=0x7efdac15be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac15be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 -- 172.21.15.22:0/1038088793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac0aa780 con 0x7efdac06f560 2024-04-27T07:51:01.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7efdb2284640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efda4007e40 tx=0x7efda400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.152+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb800578b0 con 0x7ffb8815cc70 2024-04-27T07:51:01.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2934056080 con 0x7f29440b7410 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4014620 con 0x7efdac06f560 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4019070 con 0x7efdac06f560 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4014a00 con 0x7efdac06f560 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac0aaa60 con 0x7efdac06f560 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac0ab020 con 0x7efdac06f560 2024-04-27T07:51:01.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd940052a0 con 0x7efdac06f560 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.160+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efda4014ba0 con 0x7efdac06f560 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.164+0000 7efda2ffd640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 0x7efd9006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.164+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efda4055180 con 0x7efdac06f560 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.164+0000 7efdb1a83640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 0x7efd9006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.164+0000 7efdb1a83640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 0x7efd9006a890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0ad000 tx=0x7efd9c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.172+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efda40577e0 con 0x7efdac06f560 2024-04-27T07:51:01.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.431+0000 7f3686d86640 1 -- 172.21.15.22:0/1399354389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f365c005860 con 0x7f36800ce2d0 2024-04-27T07:51:01.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f367d7fa640 1 -- 172.21.15.22:0/1399354389 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3674056480 con 0x7f36800ce2d0 2024-04-27T07:51:01.441 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 msgr2=0x7f366406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 0x7f366406a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36780ae000 tx=0x7f36780a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 msgr2=0x7f36800990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f367400c500 tx=0x7f367400c9c0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 shutdown_connections 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3664068450 0x7f366406a910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368015dba0 0x7f368014d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36800d0ce0 0x7f368009db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 --2- 172.21.15.22:0/1399354389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f36800ce2d0 0x7f36800990a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.435+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 >> 172.21.15.22:0/1399354389 conn(0x7f36800d01c0 msgr2=0x7f368006e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.439+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 shutdown_connections 2024-04-27T07:51:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.439+0000 7f366effd640 1 -- 172.21.15.22:0/1399354389 wait complete. 2024-04-27T07:51:01.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.455+0000 7f01177fe640 1 -- 172.21.15.22:0/4040136169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0110005860 con 0x7f0134070ca0 2024-04-27T07:51:01.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.455+0000 7f0129ffb640 1 -- 172.21.15.22:0/4040136169 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01240f1300 con 0x7f0134070ca0 2024-04-27T07:51:01.463 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-04-27T07:51:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 msgr2=0x7f011806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 0x7f011806a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f01340cbc00 tx=0x7f0130009190 comp rx=0 tx=0).stop 2024-04-27T07:51:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 msgr2=0x7f0134205230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 0x7f0134205230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01240ac000 tx=0x7f012401adc0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-04-27T07:51:01.465 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 shutdown_connections 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01180683b0 0x7f011806a870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f013415ea60 0x7f01340b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01340cd620 0x7f01340b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 --2- 172.21.15.22:0/4040136169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0134070ca0 0x7f0134205230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 >> 172.21.15.22:0/4040136169 conn(0x7f01340b3020 msgr2=0x7f013415dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 shutdown_connections 2024-04-27T07:51:01.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7f013b4eb640 1 -- 172.21.15.22:0/4040136169 wait complete. 2024-04-27T07:51:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.459+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb2bc068c30 con 0x7fb2bc151cf0 2024-04-27T07:51:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.463+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc990157040 con 0x7fc9900702e0 2024-04-27T07:51:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.463+0000 7fc985ffb640 1 -- 172.21.15.22:0/1359798742 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc980056c90 con 0x7fc9900702e0 2024-04-27T07:51:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.463+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/2628499435 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2b4057120 con 0x7fb2bc151cf0 2024-04-27T07:51:01.469 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:51:01.469 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 msgr2=0x7fc97406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 0x7fc97406a840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0ae000 tx=0x7fc98c004740 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 msgr2=0x7fc990157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c450 tx=0x7fc98000c910 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 shutdown_connections 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc974068380 0x7fc97406a840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9900d0c40 0x7fc990152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9900c4460 0x7fc990154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 --2- 172.21.15.22:0/1359798742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9900702e0 0x7fc990157c70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 >> 172.21.15.22:0/1359798742 conn(0x7fc9900b3020 msgr2=0x7fc9900b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 shutdown_connections 2024-04-27T07:51:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fc9975de640 1 -- 172.21.15.22:0/1359798742 wait complete. 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 msgr2=0x7fb2a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 0x7fb2a006a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80af000 tx=0x7fb2b80a8000 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 msgr2=0x7fb2bc1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400caa0 tx=0x7fb2b400cf60 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 shutdown_connections 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb2a0068380 0x7fb2a006a840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2bc151cf0 0x7fb2bc1570e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2bc0cea10 0x7fb2bc1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 --2- 172.21.15.22:0/2628499435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2bc0b5740 0x7fb2bc204140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 >> 172.21.15.22:0/2628499435 conn(0x7fb2bc067f10 msgr2=0x7fb2bc159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 shutdown_connections 2024-04-27T07:51:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7fb2c4285640 1 -- 172.21.15.22:0/2628499435 wait complete. 2024-04-27T07:51:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1b70005590 con 0x7f1b8815b5f0 2024-04-27T07:51:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.467+0000 7f1b7effd640 1 -- 172.21.15.22:0/3163833329 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b80056bc0 con 0x7f1b8815b5f0 2024-04-27T07:51:01.475 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:51:01.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 msgr2=0x7f1b6c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 0x7f1b6c06a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b840b2000 tx=0x7f1b840aa070 comp rx=0 tx=0).stop 2024-04-27T07:51:01.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 msgr2=0x7f1b8814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c710 tx=0x7f1b8000cbd0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 shutdown_connections 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1b6c0683d0 0x7f1b6c06a890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8815b5f0 0x7f1b8814afd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b880b3af0 0x7f1b880b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 --2- 172.21.15.22:0/3163833329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b880b0800 0x7f1b8809eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 >> 172.21.15.22:0/3163833329 conn(0x7f1b880b2fd0 msgr2=0x7f1b88150650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 shutdown_connections 2024-04-27T07:51:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.471+0000 7f1b8f038640 1 -- 172.21.15.22:0/3163833329 wait complete. 2024-04-27T07:51:01.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffb8815ee10 con 0x7ffb8815cc70 2024-04-27T07:51:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb76ffd640 1 -- 172.21.15.22:0/3439071473 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb800571b0 con 0x7ffb8815cc70 2024-04-27T07:51:01.483 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:51:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 msgr2=0x7ffb6806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 0x7ffb6806a890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb780c0000 tx=0x7ffb780b8000 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 msgr2=0x7ffb8814fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c760 tx=0x7ffb8000cc20 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 shutdown_connections 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ffb680683d0 0x7ffb6806a890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb8815cc70 0x7ffb8814fac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb880b4630 0x7ffb88152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 --2- 172.21.15.22:0/3439071473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb880b41d0 0x7ffb88164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 >> 172.21.15.22:0/3439071473 conn(0x7ffb880b66f0 msgr2=0x7ffb8815bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.479+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 shutdown_connections 2024-04-27T07:51:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.483+0000 7ffb8759e640 1 -- 172.21.15.22:0/3439071473 wait complete. 2024-04-27T07:51:01.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-04-27T07:51:01.493 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.491+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2928005860 con 0x7f29440b7410 2024-04-27T07:51:01.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.491+0000 7f2940ff9640 1 -- 172.21.15.22:0/3324184919 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2934055980 con 0x7f29440b7410 2024-04-27T07:51:01.498 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2024-04-27T07:51:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 msgr2=0x7f292406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 0x7f292406a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f292c0ae000 tx=0x7f292c0a7000 comp rx=0 tx=0).stop 2024-04-27T07:51:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 msgr2=0x7f29440ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f293400c450 tx=0x7f293400c910 comp rx=0 tx=0).stop 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 shutdown_connections 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2924068380 0x7f292406a840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29440d2ba0 0x7f294409f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29440d0dc0 0x7f2944203fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 --2- 172.21.15.22:0/3324184919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29440b7410 0x7f29440ce750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 >> 172.21.15.22:0/3324184919 conn(0x7f294408bfc0 msgr2=0x7f294415f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 shutdown_connections 2024-04-27T07:51:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.495+0000 7f2948c1a640 1 -- 172.21.15.22:0/3324184919 wait complete. 2024-04-27T07:51:01.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-04-27T07:51:01.503 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2024-04-27T07:51:01.504 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-04-27T07:51:01.509 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.503+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efd94005590 con 0x7efdac06f560 2024-04-27T07:51:01.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.503+0000 7efda2ffd640 1 -- 172.21.15.22:0/1038088793 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efda40570e0 con 0x7efdac06f560 2024-04-27T07:51:01.509 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-04-27T07:51:01.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-04-27T07:51:01.510 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 msgr2=0x7efd9006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 0x7efd9006a890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0ad000 tx=0x7efd9c0a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 msgr2=0x7efdac0ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efda4007e40 tx=0x7efda400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 shutdown_connections 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efd900683d0 0x7efd9006a890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdac162da0 0x7efdac0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdac0ce260 0x7efdac15be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 --2- 172.21.15.22:0/1038088793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdac06f560 0x7efdac0ad520 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 >> 172.21.15.22:0/1038088793 conn(0x7efdac067f10 msgr2=0x7efdac15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 shutdown_connections 2024-04-27T07:51:01.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.507+0000 7efdb44e8640 1 -- 172.21.15.22:0/1038088793 wait complete. 2024-04-27T07:51:01.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-04-27T07:51:01.520 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-04-27T07:51:01.528 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:01.528 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T07:51:01.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:51:01.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 Processor -- start 2024-04-27T07:51:01.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 -- start start 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd89c0b45f0 0x7fd89c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd89c0ce410 0x7fd89c162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c0b4a50 con 0x7fd89c0b1ce0 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd89c0b6790 con 0x7fd89c0b45f0 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a3c0d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd89c0b20c0 con 0x7fd89c0ce410 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd89c0b45f0 0x7fd89c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd89c0b45f0 0x7fd89c0cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40802/0 (socket says 172.21.15.22:40802) 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 -- 172.21.15.22:0/1743226678 learned_addr learned my addr 172.21.15.22:0/1743226678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a21aa640 1 --2- 172.21.15.22:0/1743226678 >> v2:172.21.15.183:3300/0 conn(0x7fd89c0ce410 0x7fd89c162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 -- 172.21.15.22:0/1743226678 >> v2:172.21.15.183:3300/0 conn(0x7fd89c0ce410 msgr2=0x7fd89c162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 --2- 172.21.15.22:0/1743226678 >> v2:172.21.15.183:3300/0 conn(0x7fd89c0ce410 0x7fd89c162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 -- 172.21.15.22:0/1743226678 >> v1:172.21.15.183:6789/0 conn(0x7fd89c0b1ce0 legacy=0x7fd89c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:01.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.675+0000 7fd8a11a8640 1 -- 172.21.15.22:0/1743226678 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c0b61d0 con 0x7fd89c0b45f0 2024-04-27T07:51:01.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a11a8640 1 --2- 172.21.15.22:0/1743226678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0b45f0 0x7fd89c0cded0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0741a0 tx=0x7fd88c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fb6b6b2aa56be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a09a7640 1 -- 172.21.15.22:0/1743226678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0b1070 con 0x7fd89c0b45f0 2024-04-27T07:51:01.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a09a7640 1 -- 172.21.15.22:0/1743226678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88c0ac620 con 0x7fd89c0b45f0 2024-04-27T07:51:01.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a09a7640 1 -- 172.21.15.22:0/1743226678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0ac9f0 con 0x7fd89c0b45f0 2024-04-27T07:51:01.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/1743226678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0b45f0 msgr2=0x7fd89c0cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/1743226678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0b45f0 0x7fd89c0cded0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0741a0 tx=0x7fd88c01c4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/1743226678 shutdown_connections 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/1743226678 >> v2:172.21.15.183:3300/0 conn(0x7fd89c0ce410 0x7fd89c162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/1743226678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0b45f0 0x7fd89c0cded0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/1743226678 >> 172.21.15.22:0/1743226678 conn(0x7fd89c09aa30 msgr2=0x7fd89c0b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/1743226678 shutdown_connections 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/1743226678 wait complete. 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 Processor -- start 2024-04-27T07:51:01.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- start start 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 0x7fd89c15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 0x7fd89c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c16bac0 con 0x7fd89c0ce410 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c16b940 con 0x7fd89c0b1ce0 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c16bc40 con 0x7fd89c0b45f0 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40812/0 (socket says 172.21.15.22:40812) 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 -- 172.21.15.22:0/4214461892 learned_addr learned my addr 172.21.15.22:0/4214461892 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a11a8640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 0x7fd89c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a19a9640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 0x7fd89c15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 -- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 msgr2=0x7fd89c13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 0x7fd89c13e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 -- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 msgr2=0x7fd89c15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 0x7fd89c15d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 -- 172.21.15.22:0/4214461892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c13ebb0 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a11a8640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 0x7fd89c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a19a9640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 0x7fd89c15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a21aa640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd894007e40 tx=0x7fd89400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894014620 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd894019070 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c15df30 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894014a20 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.679+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c15f050 con 0x7fd89c0ce410 2024-04-27T07:51:01.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c069340 con 0x7fd89c0ce410 2024-04-27T07:51:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd894014bc0 con 0x7fd89c0ce410 2024-04-27T07:51:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8927fc640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 0x7fd88006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd894087460 con 0x7fd89c0ce410 2024-04-27T07:51:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8a19a9640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 0x7fd88006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.683+0000 7fd8a19a9640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 0x7fd88006a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd89800a850 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:01.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.687+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd89c13ebb0 con 0x7fd89c0ce410 2024-04-27T07:51:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.839+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd89c09b080 con 0x7fd880068380 2024-04-27T07:51:01.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.855+0000 7fd8927fc640 1 -- 172.21.15.22:0/4214461892 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fd89c09b080 con 0x7fd880068380 2024-04-27T07:51:01.864 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:01.866 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 msgr2=0x7fd88006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 0x7fd88006a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd89800a850 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 msgr2=0x7fd89c0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd894007e40 tx=0x7fd89400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 shutdown_connections 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd880068380 0x7fd88006a840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd89c0ce410 0x7fd89c0aff20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd89c0b45f0 0x7fd89c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 --2- 172.21.15.22:0/4214461892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd89c0b1ce0 0x7fd89c15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:01.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 >> 172.21.15.22:0/4214461892 conn(0x7fd89c09aa30 msgr2=0x7fd89c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:01.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 shutdown_connections 2024-04-27T07:51:01.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:01.859+0000 7fd8a3c0d640 1 -- 172.21.15.22:0/4214461892 wait complete. 2024-04-27T07:51:01.881 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-27T07:51:00.151865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235696,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739024,"statfs":{"total":773094113280,"available":772852760576,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.355726"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283542+0000","last_change":"2024-04-27T07:50:48.807748+0000","last_active":"2024-04-27T07:50:51.283542+0000","last_peered":"2024-04-27T07:50:51.283542+0000","last_clean":"2024-04-27T07:50:51.283542+0000","last_became_active":"2024-04-27T07:50:48.806582+0000","last_became_peered":"2024-04-27T07:50:48.806582+0000","last_unstale":"2024-04-27T07:50:51.283542+0000","last_undegraded":"2024-04-27T07:50:51.283542+0000","last_fullsized":"2024-04-27T07:50:51.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.223671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282205+0000","last_change":"2024-04-27T07:50:48.809136+0000","last_active":"2024-04-27T07:50:51.282205+0000","last_peered":"2024-04-27T07:50:51.282205+0000","last_clean":"2024-04-27T07:50:51.282205+0000","last_became_active":"2024-04-27T07:50:48.808581+0000","last_became_peered":"2024-04-27T07:50:48.808581+0000","last_unstale":"2024-04-27T07:50:51.282205+0000","last_undegraded":"2024-04-27T07:50:51.282205+0000","last_fullsized":"2024-04-27T07:50:51.282205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923394+0000","last_change":"2024-04-27T07:50:48.803232+0000","last_active":"2024-04-27T07:50:50.923394+0000","last_peered":"2024-04-27T07:50:50.923394+0000","last_clean":"2024-04-27T07:50:50.923394+0000","last_became_active":"2024-04-27T07:50:48.802828+0000","last_became_peered":"2024-04-27T07:50:48.802828+0000","last_unstale":"2024-04-27T07:50:50.923394+0000","last_undegraded":"2024-04-27T07:50:50.923394+0000","last_fullsized":"2024-04-27T07:50:50.923394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800701+0000","last_change":"2024-04-27T07:50:48.822942+0000","last_active":"2024-04-27T07:50:49.800701+0000","last_peered":"2024-04-27T07:50:49.800701+0000","last_clean":"2024-04-27T07:50:49.800701+0000","last_became_active":"2024-04-27T07:50:48.822709+0000","last_became_peered":"2024-04-27T07:50:48.822709+0000","last_unstale":"2024-04-27T07:50:49.800701+0000","last_undegraded":"2024-04-27T07:50:49.800701+0000","last_fullsized":"2024-04-27T07:50:49.800701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:38.768492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283631+0000","last_change":"2024-04-27T07:50:48.805852+0000","last_active":"2024-04-27T07:50:51.283631+0000","last_peered":"2024-04-27T07:50:51.283631+0000","last_clean":"2024-04-27T07:50:51.283631+0000","last_became_active":"2024-04-27T07:50:48.805553+0000","last_became_peered":"2024-04-27T07:50:48.805553+0000","last_unstale":"2024-04-27T07:50:51.283631+0000","last_undegraded":"2024-04-27T07:50:51.283631+0000","last_fullsized":"2024-04-27T07:50:51.283631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:19:06.682886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800634+0000","last_change":"2024-04-27T07:50:48.795482+0000","last_active":"2024-04-27T07:50:49.800634+0000","last_peered":"2024-04-27T07:50:49.800634+0000","last_clean":"2024-04-27T07:50:49.800634+0000","last_became_active":"2024-04-27T07:50:48.793813+0000","last_became_peered":"2024-04-27T07:50:48.793813+0000","last_unstale":"2024-04-27T07:50:49.800634+0000","last_undegraded":"2024-04-27T07:50:49.800634+0000","last_fullsized":"2024-04-27T07:50:49.800634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:39:48.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282786+0000","last_change":"2024-04-27T07:50:48.821409+0000","last_active":"2024-04-27T07:50:51.282786+0000","last_peered":"2024-04-27T07:50:51.282786+0000","last_clean":"2024-04-27T07:50:51.282786+0000","last_became_active":"2024-04-27T07:50:48.821162+0000","last_became_peered":"2024-04-27T07:50:48.821162+0000","last_unstale":"2024-04-27T07:50:51.282786+0000","last_undegraded":"2024-04-27T07:50:51.282786+0000","last_fullsized":"2024-04-27T07:50:51.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:20:43.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281085+0000","last_change":"2024-04-27T07:50:48.830026+0000","last_active":"2024-04-27T07:50:51.281085+0000","last_peered":"2024-04-27T07:50:51.281085+0000","last_clean":"2024-04-27T07:50:51.281085+0000","last_became_active":"2024-04-27T07:50:48.821303+0000","last_became_peered":"2024-04-27T07:50:48.821303+0000","last_unstale":"2024-04-27T07:50:51.281085+0000","last_undegraded":"2024-04-27T07:50:51.281085+0000","last_fullsized":"2024-04-27T07:50:51.281085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283709+0000","last_change":"2024-04-27T07:50:48.820925+0000","last_active":"2024-04-27T07:50:51.283709+0000","last_peered":"2024-04-27T07:50:51.283709+0000","last_clean":"2024-04-27T07:50:51.283709+0000","last_became_active":"2024-04-27T07:50:48.820575+0000","last_became_peered":"2024-04-27T07:50:48.820575+0000","last_unstale":"2024-04-27T07:50:51.283709+0000","last_undegraded":"2024-04-27T07:50:51.283709+0000","last_fullsized":"2024-04-27T07:50:51.283709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:40.499571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446110+0000","last_change":"2024-04-27T07:50:48.814093+0000","last_active":"2024-04-27T07:50:50.446110+0000","last_peered":"2024-04-27T07:50:50.446110+0000","last_clean":"2024-04-27T07:50:50.446110+0000","last_became_active":"2024-04-27T07:50:48.813643+0000","last_became_peered":"2024-04-27T07:50:48.813643+0000","last_unstale":"2024-04-27T07:50:50.446110+0000","last_undegraded":"2024-04-27T07:50:50.446110+0000","last_fullsized":"2024-04-27T07:50:50.446110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:24.403447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445905+0000","last_change":"2024-04-27T07:50:48.815381+0000","last_active":"2024-04-27T07:50:50.445905+0000","last_peered":"2024-04-27T07:50:50.445905+0000","last_clean":"2024-04-27T07:50:50.445905+0000","last_became_active":"2024-04-27T07:50:48.814574+0000","last_became_peered":"2024-04-27T07:50:48.814574+0000","last_unstale":"2024-04-27T07:50:50.445905+0000","last_undegraded":"2024-04-27T07:50:50.445905+0000","last_fullsized":"2024-04-27T07:50:50.445905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:25.162778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800227+0000","last_change":"2024-04-27T07:50:48.806044+0000","last_active":"2024-04-27T07:50:49.800227+0000","last_peered":"2024-04-27T07:50:49.800227+0000","last_clean":"2024-04-27T07:50:49.800227+0000","last_became_active":"2024-04-27T07:50:48.805705+0000","last_became_peered":"2024-04-27T07:50:48.805705+0000","last_unstale":"2024-04-27T07:50:49.800227+0000","last_undegraded":"2024-04-27T07:50:49.800227+0000","last_fullsized":"2024-04-27T07:50:49.800227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:21.183002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282310+0000","last_change":"2024-04-27T07:50:48.819990+0000","last_active":"2024-04-27T07:50:51.282310+0000","last_peered":"2024-04-27T07:50:51.282310+0000","last_clean":"2024-04-27T07:50:51.282310+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.282310+0000","last_undegraded":"2024-04-27T07:50:51.282310+0000","last_fullsized":"2024-04-27T07:50:51.282310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922720+0000","last_change":"2024-04-27T07:50:48.804618+0000","last_active":"2024-04-27T07:50:50.922720+0000","last_peered":"2024-04-27T07:50:50.922720+0000","last_clean":"2024-04-27T07:50:50.922720+0000","last_became_active":"2024-04-27T07:50:48.803939+0000","last_became_peered":"2024-04-27T07:50:48.803939+0000","last_unstale":"2024-04-27T07:50:50.922720+0000","last_undegraded":"2024-04-27T07:50:50.922720+0000","last_fullsized":"2024-04-27T07:50:50.922720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:38.135106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281763+0000","last_change":"2024-04-27T07:50:48.813856+0000","last_active":"2024-04-27T07:50:51.281763+0000","last_peered":"2024-04-27T07:50:51.281763+0000","last_clean":"2024-04-27T07:50:51.281763+0000","last_became_active":"2024-04-27T07:50:48.813437+0000","last_became_peered":"2024-04-27T07:50:48.813437+0000","last_unstale":"2024-04-27T07:50:51.281763+0000","last_undegraded":"2024-04-27T07:50:51.281763+0000","last_fullsized":"2024-04-27T07:50:51.281763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801741+0000","last_change":"2024-04-27T07:50:48.810807+0000","last_active":"2024-04-27T07:50:49.801741+0000","last_peered":"2024-04-27T07:50:49.801741+0000","last_clean":"2024-04-27T07:50:49.801741+0000","last_became_active":"2024-04-27T07:50:48.810545+0000","last_became_peered":"2024-04-27T07:50:48.810545+0000","last_unstale":"2024-04-27T07:50:49.801741+0000","last_undegraded":"2024-04-27T07:50:49.801741+0000","last_fullsized":"2024-04-27T07:50:49.801741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:09.348372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799832+0000","last_change":"2024-04-27T07:50:48.801260+0000","last_active":"2024-04-27T07:50:49.799832+0000","last_peered":"2024-04-27T07:50:49.799832+0000","last_clean":"2024-04-27T07:50:49.799832+0000","last_became_active":"2024-04-27T07:50:48.800565+0000","last_became_peered":"2024-04-27T07:50:48.800565+0000","last_unstale":"2024-04-27T07:50:49.799832+0000","last_undegraded":"2024-04-27T07:50:49.799832+0000","last_fullsized":"2024-04-27T07:50:49.799832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797791+0000","last_change":"2024-04-27T07:50:48.806722+0000","last_active":"2024-04-27T07:50:49.797791+0000","last_peered":"2024-04-27T07:50:49.797791+0000","last_clean":"2024-04-27T07:50:49.797791+0000","last_became_active":"2024-04-27T07:50:48.805899+0000","last_became_peered":"2024-04-27T07:50:48.805899+0000","last_unstale":"2024-04-27T07:50:49.797791+0000","last_undegraded":"2024-04-27T07:50:49.797791+0000","last_fullsized":"2024-04-27T07:50:49.797791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:28:00.765509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283852+0000","last_change":"2024-04-27T07:50:48.806262+0000","last_active":"2024-04-27T07:50:51.283852+0000","last_peered":"2024-04-27T07:50:51.283852+0000","last_clean":"2024-04-27T07:50:51.283852+0000","last_became_active":"2024-04-27T07:50:48.805987+0000","last_became_peered":"2024-04-27T07:50:48.805987+0000","last_unstale":"2024-04-27T07:50:51.283852+0000","last_undegraded":"2024-04-27T07:50:51.283852+0000","last_fullsized":"2024-04-27T07:50:51.283852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:49:40.263257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445516+0000","last_change":"2024-04-27T07:50:48.811277+0000","last_active":"2024-04-27T07:50:50.445516+0000","last_peered":"2024-04-27T07:50:50.445516+0000","last_clean":"2024-04-27T07:50:50.445516+0000","last_became_active":"2024-04-27T07:50:48.810296+0000","last_became_peered":"2024-04-27T07:50:48.810296+0000","last_unstale":"2024-04-27T07:50:50.445516+0000","last_undegraded":"2024-04-27T07:50:50.445516+0000","last_fullsized":"2024-04-27T07:50:50.445516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:43:55.556626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922970+0000","last_change":"2024-04-27T07:50:48.798022+0000","last_active":"2024-04-27T07:50:50.922970+0000","last_peered":"2024-04-27T07:50:50.922970+0000","last_clean":"2024-04-27T07:50:50.922970+0000","last_became_active":"2024-04-27T07:50:48.797464+0000","last_became_peered":"2024-04-27T07:50:48.797464+0000","last_unstale":"2024-04-27T07:50:50.922970+0000","last_undegraded":"2024-04-27T07:50:50.922970+0000","last_fullsized":"2024-04-27T07:50:50.922970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:39:13.996688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800758+0000","last_change":"2024-04-27T07:50:48.796045+0000","last_active":"2024-04-27T07:50:49.800758+0000","last_peered":"2024-04-27T07:50:49.800758+0000","last_clean":"2024-04-27T07:50:49.800758+0000","last_became_active":"2024-04-27T07:50:48.794825+0000","last_became_peered":"2024-04-27T07:50:48.794825+0000","last_unstale":"2024-04-27T07:50:49.800758+0000","last_undegraded":"2024-04-27T07:50:49.800758+0000","last_fullsized":"2024-04-27T07:50:49.800758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:49.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281332+0000","last_change":"2024-04-27T07:50:48.821433+0000","last_active":"2024-04-27T07:50:51.281332+0000","last_peered":"2024-04-27T07:50:51.281332+0000","last_clean":"2024-04-27T07:50:51.281332+0000","last_became_active":"2024-04-27T07:50:48.820660+0000","last_became_peered":"2024-04-27T07:50:48.820660+0000","last_unstale":"2024-04-27T07:50:51.281332+0000","last_undegraded":"2024-04-27T07:50:51.281332+0000","last_fullsized":"2024-04-27T07:50:51.281332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799368+0000","last_change":"2024-04-27T07:50:48.810201+0000","last_active":"2024-04-27T07:50:49.799368+0000","last_peered":"2024-04-27T07:50:49.799368+0000","last_clean":"2024-04-27T07:50:49.799368+0000","last_became_active":"2024-04-27T07:50:48.809937+0000","last_became_peered":"2024-04-27T07:50:48.809937+0000","last_unstale":"2024-04-27T07:50:49.799368+0000","last_undegraded":"2024-04-27T07:50:49.799368+0000","last_fullsized":"2024-04-27T07:50:49.799368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283175+0000","last_change":"2024-04-27T07:50:48.810567+0000","last_active":"2024-04-27T07:50:51.283175+0000","last_peered":"2024-04-27T07:50:51.283175+0000","last_clean":"2024-04-27T07:50:51.283175+0000","last_became_active":"2024-04-27T07:50:48.809921+0000","last_became_peered":"2024-04-27T07:50:48.809921+0000","last_unstale":"2024-04-27T07:50:51.283175+0000","last_undegraded":"2024-04-27T07:50:51.283175+0000","last_fullsized":"2024-04-27T07:50:51.283175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446226+0000","last_change":"2024-04-27T07:50:48.805860+0000","last_active":"2024-04-27T07:50:50.446226+0000","last_peered":"2024-04-27T07:50:50.446226+0000","last_clean":"2024-04-27T07:50:50.446226+0000","last_became_active":"2024-04-27T07:50:48.805182+0000","last_became_peered":"2024-04-27T07:50:48.805182+0000","last_unstale":"2024-04-27T07:50:50.446226+0000","last_undegraded":"2024-04-27T07:50:50.446226+0000","last_fullsized":"2024-04-27T07:50:50.446226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:10:53.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281842+0000","last_change":"2024-04-27T07:50:48.811553+0000","last_active":"2024-04-27T07:50:51.281842+0000","last_peered":"2024-04-27T07:50:51.281842+0000","last_clean":"2024-04-27T07:50:51.281842+0000","last_became_active":"2024-04-27T07:50:48.810935+0000","last_became_peered":"2024-04-27T07:50:48.810935+0000","last_unstale":"2024-04-27T07:50:51.281842+0000","last_undegraded":"2024-04-27T07:50:51.281842+0000","last_fullsized":"2024-04-27T07:50:51.281842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283139+0000","last_change":"2024-04-27T07:50:48.820547+0000","last_active":"2024-04-27T07:50:51.283139+0000","last_peered":"2024-04-27T07:50:51.283139+0000","last_clean":"2024-04-27T07:50:51.283139+0000","last_became_active":"2024-04-27T07:50:48.820170+0000","last_became_peered":"2024-04-27T07:50:48.820170+0000","last_unstale":"2024-04-27T07:50:51.283139+0000","last_undegraded":"2024-04-27T07:50:51.283139+0000","last_fullsized":"2024-04-27T07:50:51.283139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923336+0000","last_change":"2024-04-27T07:50:48.821676+0000","last_active":"2024-04-27T07:50:50.923336+0000","last_peered":"2024-04-27T07:50:50.923336+0000","last_clean":"2024-04-27T07:50:50.923336+0000","last_became_active":"2024-04-27T07:50:48.820884+0000","last_became_peered":"2024-04-27T07:50:48.820884+0000","last_unstale":"2024-04-27T07:50:50.923336+0000","last_undegraded":"2024-04-27T07:50:50.923336+0000","last_fullsized":"2024-04-27T07:50:50.923336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800884+0000","last_change":"2024-04-27T07:50:48.808706+0000","last_active":"2024-04-27T07:50:49.800884+0000","last_peered":"2024-04-27T07:50:49.800884+0000","last_clean":"2024-04-27T07:50:49.800884+0000","last_became_active":"2024-04-27T07:50:48.804475+0000","last_became_peered":"2024-04-27T07:50:48.804475+0000","last_unstale":"2024-04-27T07:50:49.800884+0000","last_undegraded":"2024-04-27T07:50:49.800884+0000","last_fullsized":"2024-04-27T07:50:49.800884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:07:50.589385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797415+0000","last_change":"2024-04-27T07:50:48.810016+0000","last_active":"2024-04-27T07:50:49.797415+0000","last_peered":"2024-04-27T07:50:49.797415+0000","last_clean":"2024-04-27T07:50:49.797415+0000","last_became_active":"2024-04-27T07:50:48.809618+0000","last_became_peered":"2024-04-27T07:50:48.809618+0000","last_unstale":"2024-04-27T07:50:49.797415+0000","last_undegraded":"2024-04-27T07:50:49.797415+0000","last_fullsized":"2024-04-27T07:50:49.797415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:20.530307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445841+0000","last_change":"2024-04-27T07:50:48.805951+0000","last_active":"2024-04-27T07:50:50.445841+0000","last_peered":"2024-04-27T07:50:50.445841+0000","last_clean":"2024-04-27T07:50:50.445841+0000","last_became_active":"2024-04-27T07:50:48.805493+0000","last_became_peered":"2024-04-27T07:50:48.805493+0000","last_unstale":"2024-04-27T07:50:50.445841+0000","last_undegraded":"2024-04-27T07:50:50.445841+0000","last_fullsized":"2024-04-27T07:50:50.445841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:42:36.766605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800350+0000","last_change":"2024-04-27T07:50:48.799052+0000","last_active":"2024-04-27T07:50:49.800350+0000","last_peered":"2024-04-27T07:50:49.800350+0000","last_clean":"2024-04-27T07:50:49.800350+0000","last_became_active":"2024-04-27T07:50:48.798769+0000","last_became_peered":"2024-04-27T07:50:48.798769+0000","last_unstale":"2024-04-27T07:50:49.800350+0000","last_undegraded":"2024-04-27T07:50:49.800350+0000","last_fullsized":"2024-04-27T07:50:49.800350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:19.179469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282661+0000","last_change":"2024-04-27T07:50:47.787481+0000","last_active":"2024-04-27T07:50:51.282661+0000","last_peered":"2024-04-27T07:50:51.282661+0000","last_clean":"2024-04-27T07:50:51.282661+0000","last_became_active":"2024-04-27T07:50:47.787133+0000","last_became_peered":"2024-04-27T07:50:47.787133+0000","last_unstale":"2024-04-27T07:50:51.282661+0000","last_undegraded":"2024-04-27T07:50:51.282661+0000","last_fullsized":"2024-04-27T07:50:51.282661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:50.164872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923143+0000","last_change":"2024-04-27T07:50:48.812526+0000","last_active":"2024-04-27T07:50:50.923143+0000","last_peered":"2024-04-27T07:50:50.923143+0000","last_clean":"2024-04-27T07:50:50.923143+0000","last_became_active":"2024-04-27T07:50:48.812099+0000","last_became_peered":"2024-04-27T07:50:48.812099+0000","last_unstale":"2024-04-27T07:50:50.923143+0000","last_undegraded":"2024-04-27T07:50:50.923143+0000","last_fullsized":"2024-04-27T07:50:50.923143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445014+0000","last_change":"2024-04-27T07:50:47.786268+0000","last_active":"2024-04-27T07:50:50.445014+0000","last_peered":"2024-04-27T07:50:50.445014+0000","last_clean":"2024-04-27T07:50:50.445014+0000","last_became_active":"2024-04-27T07:50:47.785467+0000","last_became_peered":"2024-04-27T07:50:47.785467+0000","last_unstale":"2024-04-27T07:50:50.445014+0000","last_undegraded":"2024-04-27T07:50:50.445014+0000","last_fullsized":"2024-04-27T07:50:50.445014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283343+0000","last_change":"2024-04-27T07:50:47.782977+0000","last_active":"2024-04-27T07:50:51.283343+0000","last_peered":"2024-04-27T07:50:51.283343+0000","last_clean":"2024-04-27T07:50:51.283343+0000","last_became_active":"2024-04-27T07:50:47.782680+0000","last_became_peered":"2024-04-27T07:50:47.782680+0000","last_unstale":"2024-04-27T07:50:51.283343+0000","last_undegraded":"2024-04-27T07:50:51.283343+0000","last_fullsized":"2024-04-27T07:50:51.283343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281283+0000","last_change":"2024-04-27T07:50:48.795764+0000","last_active":"2024-04-27T07:50:51.281283+0000","last_peered":"2024-04-27T07:50:51.281283+0000","last_clean":"2024-04-27T07:50:51.281283+0000","last_became_active":"2024-04-27T07:50:48.794946+0000","last_became_peered":"2024-04-27T07:50:48.794946+0000","last_unstale":"2024-04-27T07:50:51.281283+0000","last_undegraded":"2024-04-27T07:50:51.281283+0000","last_fullsized":"2024-04-27T07:50:51.281283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799501+0000","last_change":"2024-04-27T07:50:47.768634+0000","last_active":"2024-04-27T07:50:49.799501+0000","last_peered":"2024-04-27T07:50:49.799501+0000","last_clean":"2024-04-27T07:50:49.799501+0000","last_became_active":"2024-04-27T07:50:47.767344+0000","last_became_peered":"2024-04-27T07:50:47.767344+0000","last_unstale":"2024-04-27T07:50:49.799501+0000","last_undegraded":"2024-04-27T07:50:49.799501+0000","last_fullsized":"2024-04-27T07:50:49.799501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281548+0000","last_change":"2024-04-27T07:50:48.831910+0000","last_active":"2024-04-27T07:50:51.281548+0000","last_peered":"2024-04-27T07:50:51.281548+0000","last_clean":"2024-04-27T07:50:51.281548+0000","last_became_active":"2024-04-27T07:50:48.831339+0000","last_became_peered":"2024-04-27T07:50:48.831339+0000","last_unstale":"2024-04-27T07:50:51.281548+0000","last_undegraded":"2024-04-27T07:50:51.281548+0000","last_fullsized":"2024-04-27T07:50:51.281548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445402+0000","last_change":"2024-04-27T07:50:47.784442+0000","last_active":"2024-04-27T07:50:50.445402+0000","last_peered":"2024-04-27T07:50:50.445402+0000","last_clean":"2024-04-27T07:50:50.445402+0000","last_became_active":"2024-04-27T07:50:47.783769+0000","last_became_peered":"2024-04-27T07:50:47.783769+0000","last_unstale":"2024-04-27T07:50:50.445402+0000","last_undegraded":"2024-04-27T07:50:50.445402+0000","last_fullsized":"2024-04-27T07:50:50.445402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:15.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799300+0000","last_change":"2024-04-27T07:50:48.806497+0000","last_active":"2024-04-27T07:50:49.799300+0000","last_peered":"2024-04-27T07:50:49.799300+0000","last_clean":"2024-04-27T07:50:49.799300+0000","last_became_active":"2024-04-27T07:50:48.806172+0000","last_became_peered":"2024-04-27T07:50:48.806172+0000","last_unstale":"2024-04-27T07:50:49.799300+0000","last_undegraded":"2024-04-27T07:50:49.799300+0000","last_fullsized":"2024-04-27T07:50:49.799300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799348+0000","last_change":"2024-04-27T07:50:47.776964+0000","last_active":"2024-04-27T07:50:49.799348+0000","last_peered":"2024-04-27T07:50:49.799348+0000","last_clean":"2024-04-27T07:50:49.799348+0000","last_became_active":"2024-04-27T07:50:47.776179+0000","last_became_peered":"2024-04-27T07:50:47.776179+0000","last_unstale":"2024-04-27T07:50:49.799348+0000","last_undegraded":"2024-04-27T07:50:49.799348+0000","last_fullsized":"2024-04-27T07:50:49.799348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923673+0000","last_change":"2024-04-27T07:50:48.812739+0000","last_active":"2024-04-27T07:50:50.923673+0000","last_peered":"2024-04-27T07:50:50.923673+0000","last_clean":"2024-04-27T07:50:50.923673+0000","last_became_active":"2024-04-27T07:50:48.812469+0000","last_became_peered":"2024-04-27T07:50:48.812469+0000","last_unstale":"2024-04-27T07:50:50.923673+0000","last_undegraded":"2024-04-27T07:50:50.923673+0000","last_fullsized":"2024-04-27T07:50:50.923673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281980+0000","last_change":"2024-04-27T07:50:47.774587+0000","last_active":"2024-04-27T07:50:51.281980+0000","last_peered":"2024-04-27T07:50:51.281980+0000","last_clean":"2024-04-27T07:50:51.281980+0000","last_became_active":"2024-04-27T07:50:47.769083+0000","last_became_peered":"2024-04-27T07:50:47.769083+0000","last_unstale":"2024-04-27T07:50:51.281980+0000","last_undegraded":"2024-04-27T07:50:51.281980+0000","last_fullsized":"2024-04-27T07:50:51.281980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800330+0000","last_change":"2024-04-27T07:50:48.798248+0000","last_active":"2024-04-27T07:50:49.800330+0000","last_peered":"2024-04-27T07:50:49.800330+0000","last_clean":"2024-04-27T07:50:49.800330+0000","last_became_active":"2024-04-27T07:50:48.797987+0000","last_became_peered":"2024-04-27T07:50:48.797987+0000","last_unstale":"2024-04-27T07:50:49.800330+0000","last_undegraded":"2024-04-27T07:50:49.800330+0000","last_fullsized":"2024-04-27T07:50:49.800330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:28.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445367+0000","last_change":"2024-04-27T07:50:47.783980+0000","last_active":"2024-04-27T07:50:50.445367+0000","last_peered":"2024-04-27T07:50:50.445367+0000","last_clean":"2024-04-27T07:50:50.445367+0000","last_became_active":"2024-04-27T07:50:47.783127+0000","last_became_peered":"2024-04-27T07:50:47.783127+0000","last_unstale":"2024-04-27T07:50:50.445367+0000","last_undegraded":"2024-04-27T07:50:50.445367+0000","last_fullsized":"2024-04-27T07:50:50.445367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283118+0000","last_change":"2024-04-27T07:50:48.822749+0000","last_active":"2024-04-27T07:50:51.283118+0000","last_peered":"2024-04-27T07:50:51.283118+0000","last_clean":"2024-04-27T07:50:51.283118+0000","last_became_active":"2024-04-27T07:50:48.822309+0000","last_became_peered":"2024-04-27T07:50:48.822309+0000","last_unstale":"2024-04-27T07:50:51.283118+0000","last_undegraded":"2024-04-27T07:50:51.283118+0000","last_fullsized":"2024-04-27T07:50:51.283118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:43.311205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923198+0000","last_change":"2024-04-27T07:50:47.783527+0000","last_active":"2024-04-27T07:50:50.923198+0000","last_peered":"2024-04-27T07:50:50.923198+0000","last_clean":"2024-04-27T07:50:50.923198+0000","last_became_active":"2024-04-27T07:50:47.782770+0000","last_became_peered":"2024-04-27T07:50:47.782770+0000","last_unstale":"2024-04-27T07:50:50.923198+0000","last_undegraded":"2024-04-27T07:50:50.923198+0000","last_fullsized":"2024-04-27T07:50:50.923198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:30:37.164246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923078+0000","last_change":"2024-04-27T07:50:48.797984+0000","last_active":"2024-04-27T07:50:50.923078+0000","last_peered":"2024-04-27T07:50:50.923078+0000","last_clean":"2024-04-27T07:50:50.923078+0000","last_became_active":"2024-04-27T07:50:48.797194+0000","last_became_peered":"2024-04-27T07:50:48.797194+0000","last_unstale":"2024-04-27T07:50:50.923078+0000","last_undegraded":"2024-04-27T07:50:50.923078+0000","last_fullsized":"2024-04-27T07:50:50.923078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800940+0000","last_change":"2024-04-27T07:50:47.790007+0000","last_active":"2024-04-27T07:50:49.800940+0000","last_peered":"2024-04-27T07:50:49.800940+0000","last_clean":"2024-04-27T07:50:49.800940+0000","last_became_active":"2024-04-27T07:50:47.789403+0000","last_became_peered":"2024-04-27T07:50:47.789403+0000","last_unstale":"2024-04-27T07:50:49.800940+0000","last_undegraded":"2024-04-27T07:50:49.800940+0000","last_fullsized":"2024-04-27T07:50:49.800940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283254+0000","last_change":"2024-04-27T07:50:48.810819+0000","last_active":"2024-04-27T07:50:51.283254+0000","last_peered":"2024-04-27T07:50:51.283254+0000","last_clean":"2024-04-27T07:50:51.283254+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.283254+0000","last_undegraded":"2024-04-27T07:50:51.283254+0000","last_fullsized":"2024-04-27T07:50:51.283254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:50.867045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.444990+0000","last_change":"2024-04-27T07:50:47.777426+0000","last_active":"2024-04-27T07:50:50.444990+0000","last_peered":"2024-04-27T07:50:50.444990+0000","last_clean":"2024-04-27T07:50:50.444990+0000","last_became_active":"2024-04-27T07:50:47.776724+0000","last_became_peered":"2024-04-27T07:50:47.776724+0000","last_unstale":"2024-04-27T07:50:50.444990+0000","last_undegraded":"2024-04-27T07:50:50.444990+0000","last_fullsized":"2024-04-27T07:50:50.444990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:37:47.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445715+0000","last_change":"2024-04-27T07:50:48.793186+0000","last_active":"2024-04-27T07:50:50.445715+0000","last_peered":"2024-04-27T07:50:50.445715+0000","last_clean":"2024-04-27T07:50:50.445715+0000","last_became_active":"2024-04-27T07:50:48.792919+0000","last_became_peered":"2024-04-27T07:50:48.792919+0000","last_unstale":"2024-04-27T07:50:50.445715+0000","last_undegraded":"2024-04-27T07:50:50.445715+0000","last_fullsized":"2024-04-27T07:50:50.445715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:34:56.089422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797034+0000","last_change":"2024-04-27T07:50:47.770430+0000","last_active":"2024-04-27T07:50:49.797034+0000","last_peered":"2024-04-27T07:50:49.797034+0000","last_clean":"2024-04-27T07:50:49.797034+0000","last_became_active":"2024-04-27T07:50:47.770009+0000","last_became_peered":"2024-04-27T07:50:47.770009+0000","last_unstale":"2024-04-27T07:50:49.797034+0000","last_undegraded":"2024-04-27T07:50:49.797034+0000","last_fullsized":"2024-04-27T07:50:49.797034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282964+0000","last_change":"2024-04-27T07:50:48.817834+0000","last_active":"2024-04-27T07:50:51.282964+0000","last_peered":"2024-04-27T07:50:51.282964+0000","last_clean":"2024-04-27T07:50:51.282964+0000","last_became_active":"2024-04-27T07:50:48.817436+0000","last_became_peered":"2024-04-27T07:50:48.817436+0000","last_unstale":"2024-04-27T07:50:51.282964+0000","last_undegraded":"2024-04-27T07:50:51.282964+0000","last_fullsized":"2024-04-27T07:50:51.282964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:51.766684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801830+0000","last_change":"2024-04-27T07:50:47.776874+0000","last_active":"2024-04-27T07:50:49.801830+0000","last_peered":"2024-04-27T07:50:49.801830+0000","last_clean":"2024-04-27T07:50:49.801830+0000","last_became_active":"2024-04-27T07:50:47.776056+0000","last_became_peered":"2024-04-27T07:50:47.776056+0000","last_unstale":"2024-04-27T07:50:49.801830+0000","last_undegraded":"2024-04-27T07:50:49.801830+0000","last_fullsized":"2024-04-27T07:50:49.801830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:13:34.610739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283821+0000","last_change":"2024-04-27T07:50:48.812568+0000","last_active":"2024-04-27T07:50:51.283821+0000","last_peered":"2024-04-27T07:50:51.283821+0000","last_clean":"2024-04-27T07:50:51.283821+0000","last_became_active":"2024-04-27T07:50:48.811223+0000","last_became_peered":"2024-04-27T07:50:48.811223+0000","last_unstale":"2024-04-27T07:50:51.283821+0000","last_undegraded":"2024-04-27T07:50:51.283821+0000","last_fullsized":"2024-04-27T07:50:51.283821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:21:47.632417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797924+0000","last_change":"2024-04-27T07:50:47.757707+0000","last_active":"2024-04-27T07:50:49.797924+0000","last_peered":"2024-04-27T07:50:49.797924+0000","last_clean":"2024-04-27T07:50:49.797924+0000","last_became_active":"2024-04-27T07:50:47.757001+0000","last_became_peered":"2024-04-27T07:50:47.757001+0000","last_unstale":"2024-04-27T07:50:49.797924+0000","last_undegraded":"2024-04-27T07:50:49.797924+0000","last_fullsized":"2024-04-27T07:50:49.797924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283932+0000","last_change":"2024-04-27T07:50:48.820601+0000","last_active":"2024-04-27T07:50:51.283932+0000","last_peered":"2024-04-27T07:50:51.283932+0000","last_clean":"2024-04-27T07:50:51.283932+0000","last_became_active":"2024-04-27T07:50:48.817456+0000","last_became_peered":"2024-04-27T07:50:48.817456+0000","last_unstale":"2024-04-27T07:50:51.283932+0000","last_undegraded":"2024-04-27T07:50:51.283932+0000","last_fullsized":"2024-04-27T07:50:51.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:14:53.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282269+0000","last_change":"2024-04-27T07:50:47.779473+0000","last_active":"2024-04-27T07:50:51.282269+0000","last_peered":"2024-04-27T07:50:51.282269+0000","last_clean":"2024-04-27T07:50:51.282269+0000","last_became_active":"2024-04-27T07:50:47.779210+0000","last_became_peered":"2024-04-27T07:50:47.779210+0000","last_unstale":"2024-04-27T07:50:51.282269+0000","last_undegraded":"2024-04-27T07:50:51.282269+0000","last_fullsized":"2024-04-27T07:50:51.282269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801103+0000","last_change":"2024-04-27T07:50:48.822640+0000","last_active":"2024-04-27T07:50:49.801103+0000","last_peered":"2024-04-27T07:50:49.801103+0000","last_clean":"2024-04-27T07:50:49.801103+0000","last_became_active":"2024-04-27T07:50:48.822329+0000","last_became_peered":"2024-04-27T07:50:48.822329+0000","last_unstale":"2024-04-27T07:50:49.801103+0000","last_undegraded":"2024-04-27T07:50:49.801103+0000","last_fullsized":"2024-04-27T07:50:49.801103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:29.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797405+0000","last_change":"2024-04-27T07:50:47.767538+0000","last_active":"2024-04-27T07:50:49.797405+0000","last_peered":"2024-04-27T07:50:49.797405+0000","last_clean":"2024-04-27T07:50:49.797405+0000","last_became_active":"2024-04-27T07:50:47.767085+0000","last_became_peered":"2024-04-27T07:50:47.767085+0000","last_unstale":"2024-04-27T07:50:49.797405+0000","last_undegraded":"2024-04-27T07:50:49.797405+0000","last_fullsized":"2024-04-27T07:50:49.797405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283419+0000","last_change":"2024-04-27T07:50:48.818495+0000","last_active":"2024-04-27T07:50:51.283419+0000","last_peered":"2024-04-27T07:50:51.283419+0000","last_clean":"2024-04-27T07:50:51.283419+0000","last_became_active":"2024-04-27T07:50:48.817986+0000","last_became_peered":"2024-04-27T07:50:48.817986+0000","last_unstale":"2024-04-27T07:50:51.283419+0000","last_undegraded":"2024-04-27T07:50:51.283419+0000","last_fullsized":"2024-04-27T07:50:51.283419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:39:30.392854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282765+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:51.282765+0000","last_peered":"2024-04-27T07:50:51.282765+0000","last_clean":"2024-04-27T07:50:51.282765+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:51.282765+0000","last_undegraded":"2024-04-27T07:50:51.282765+0000","last_fullsized":"2024-04-27T07:50:51.282765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.798836+0000","last_change":"2024-04-27T07:50:47.765622+0000","last_active":"2024-04-27T07:50:49.798836+0000","last_peered":"2024-04-27T07:50:49.798836+0000","last_clean":"2024-04-27T07:50:49.798836+0000","last_became_active":"2024-04-27T07:50:47.764321+0000","last_became_peered":"2024-04-27T07:50:47.764321+0000","last_unstale":"2024-04-27T07:50:49.798836+0000","last_undegraded":"2024-04-27T07:50:49.798836+0000","last_fullsized":"2024-04-27T07:50:49.798836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282843+0000","last_change":"2024-04-27T07:50:48.829189+0000","last_active":"2024-04-27T07:50:51.282843+0000","last_peered":"2024-04-27T07:50:51.282843+0000","last_clean":"2024-04-27T07:50:51.282843+0000","last_became_active":"2024-04-27T07:50:48.828497+0000","last_became_peered":"2024-04-27T07:50:48.828497+0000","last_unstale":"2024-04-27T07:50:51.282843+0000","last_undegraded":"2024-04-27T07:50:51.282843+0000","last_fullsized":"2024-04-27T07:50:51.282843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282400+0000","last_change":"2024-04-27T07:50:47.801435+0000","last_active":"2024-04-27T07:50:51.282400+0000","last_peered":"2024-04-27T07:50:51.282400+0000","last_clean":"2024-04-27T07:50:51.282400+0000","last_became_active":"2024-04-27T07:50:47.800976+0000","last_became_peered":"2024-04-27T07:50:47.800976+0000","last_unstale":"2024-04-27T07:50:51.282400+0000","last_undegraded":"2024-04-27T07:50:51.282400+0000","last_fullsized":"2024-04-27T07:50:51.282400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922633+0000","last_change":"2024-04-27T07:50:48.813418+0000","last_active":"2024-04-27T07:50:50.922633+0000","last_peered":"2024-04-27T07:50:50.922633+0000","last_clean":"2024-04-27T07:50:50.922633+0000","last_became_active":"2024-04-27T07:50:48.812995+0000","last_became_peered":"2024-04-27T07:50:48.812995+0000","last_unstale":"2024-04-27T07:50:50.922633+0000","last_undegraded":"2024-04-27T07:50:50.922633+0000","last_fullsized":"2024-04-27T07:50:50.922633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:09:23.204607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797401+0000","last_change":"2024-04-27T07:50:47.753103+0000","last_active":"2024-04-27T07:50:49.797401+0000","last_peered":"2024-04-27T07:50:49.797401+0000","last_clean":"2024-04-27T07:50:49.797401+0000","last_became_active":"2024-04-27T07:50:47.752481+0000","last_became_peered":"2024-04-27T07:50:47.752481+0000","last_unstale":"2024-04-27T07:50:49.797401+0000","last_undegraded":"2024-04-27T07:50:49.797401+0000","last_fullsized":"2024-04-27T07:50:49.797401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281299+0000","last_change":"2024-04-27T07:50:48.806850+0000","last_active":"2024-04-27T07:50:51.281299+0000","last_peered":"2024-04-27T07:50:51.281299+0000","last_clean":"2024-04-27T07:50:51.281299+0000","last_became_active":"2024-04-27T07:50:48.801965+0000","last_became_peered":"2024-04-27T07:50:48.801965+0000","last_unstale":"2024-04-27T07:50:51.281299+0000","last_undegraded":"2024-04-27T07:50:51.281299+0000","last_fullsized":"2024-04-27T07:50:51.281299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801000+0000","last_change":"2024-04-27T07:50:47.779751+0000","last_active":"2024-04-27T07:50:49.801000+0000","last_peered":"2024-04-27T07:50:49.801000+0000","last_clean":"2024-04-27T07:50:49.801000+0000","last_became_active":"2024-04-27T07:50:47.778788+0000","last_became_peered":"2024-04-27T07:50:47.778788+0000","last_unstale":"2024-04-27T07:50:49.801000+0000","last_undegraded":"2024-04-27T07:50:49.801000+0000","last_fullsized":"2024-04-27T07:50:49.801000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801821+0000","last_change":"2024-04-27T07:50:48.821201+0000","last_active":"2024-04-27T07:50:49.801821+0000","last_peered":"2024-04-27T07:50:49.801821+0000","last_clean":"2024-04-27T07:50:49.801821+0000","last_became_active":"2024-04-27T07:50:48.820923+0000","last_became_peered":"2024-04-27T07:50:48.820923+0000","last_unstale":"2024-04-27T07:50:49.801821+0000","last_undegraded":"2024-04-27T07:50:49.801821+0000","last_fullsized":"2024-04-27T07:50:49.801821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:25.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281927+0000","last_change":"2024-04-27T07:50:47.769907+0000","last_active":"2024-04-27T07:50:51.281927+0000","last_peered":"2024-04-27T07:50:51.281927+0000","last_clean":"2024-04-27T07:50:51.281927+0000","last_became_active":"2024-04-27T07:50:47.769126+0000","last_became_peered":"2024-04-27T07:50:47.769126+0000","last_unstale":"2024-04-27T07:50:51.281927+0000","last_undegraded":"2024-04-27T07:50:51.281927+0000","last_fullsized":"2024-04-27T07:50:51.281927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800281+0000","last_change":"2024-04-27T07:50:48.804829+0000","last_active":"2024-04-27T07:50:49.800281+0000","last_peered":"2024-04-27T07:50:49.800281+0000","last_clean":"2024-04-27T07:50:49.800281+0000","last_became_active":"2024-04-27T07:50:48.804092+0000","last_became_peered":"2024-04-27T07:50:48.804092+0000","last_unstale":"2024-04-27T07:50:49.800281+0000","last_undegraded":"2024-04-27T07:50:49.800281+0000","last_fullsized":"2024-04-27T07:50:49.800281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283754+0000","last_change":"2024-04-27T07:50:47.801329+0000","last_active":"2024-04-27T07:50:51.283754+0000","last_peered":"2024-04-27T07:50:51.283754+0000","last_clean":"2024-04-27T07:50:51.283754+0000","last_became_active":"2024-04-27T07:50:47.800993+0000","last_became_peered":"2024-04-27T07:50:47.800993+0000","last_unstale":"2024-04-27T07:50:51.283754+0000","last_undegraded":"2024-04-27T07:50:51.283754+0000","last_fullsized":"2024-04-27T07:50:51.283754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:52:57.849530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283493+0000","last_change":"2024-04-27T07:50:48.795907+0000","last_active":"2024-04-27T07:50:51.283493+0000","last_peered":"2024-04-27T07:50:51.283493+0000","last_clean":"2024-04-27T07:50:51.283493+0000","last_became_active":"2024-04-27T07:50:48.795249+0000","last_became_peered":"2024-04-27T07:50:48.795249+0000","last_unstale":"2024-04-27T07:50:51.283493+0000","last_undegraded":"2024-04-27T07:50:51.283493+0000","last_fullsized":"2024-04-27T07:50:51.283493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282216+0000","last_change":"2024-04-27T07:50:47.774585+0000","last_active":"2024-04-27T07:50:51.282216+0000","last_peered":"2024-04-27T07:50:51.282216+0000","last_clean":"2024-04-27T07:50:51.282216+0000","last_became_active":"2024-04-27T07:50:47.774213+0000","last_became_peered":"2024-04-27T07:50:47.774213+0000","last_unstale":"2024-04-27T07:50:51.282216+0000","last_undegraded":"2024-04-27T07:50:51.282216+0000","last_fullsized":"2024-04-27T07:50:51.282216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283467+0000","last_change":"2024-04-27T07:50:48.797717+0000","last_active":"2024-04-27T07:50:51.283467+0000","last_peered":"2024-04-27T07:50:51.283467+0000","last_clean":"2024-04-27T07:50:51.283467+0000","last_became_active":"2024-04-27T07:50:48.797011+0000","last_became_peered":"2024-04-27T07:50:48.797011+0000","last_unstale":"2024-04-27T07:50:51.283467+0000","last_undegraded":"2024-04-27T07:50:51.283467+0000","last_fullsized":"2024-04-27T07:50:51.283467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445562+0000","last_change":"2024-04-27T07:50:47.796196+0000","last_active":"2024-04-27T07:50:50.445562+0000","last_peered":"2024-04-27T07:50:50.445562+0000","last_clean":"2024-04-27T07:50:50.445562+0000","last_became_active":"2024-04-27T07:50:47.795881+0000","last_became_peered":"2024-04-27T07:50:47.795881+0000","last_unstale":"2024-04-27T07:50:50.445562+0000","last_undegraded":"2024-04-27T07:50:50.445562+0000","last_fullsized":"2024-04-27T07:50:50.445562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:30:11.967127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923685+0000","last_change":"2024-04-27T07:50:48.804448+0000","last_active":"2024-04-27T07:50:50.923685+0000","last_peered":"2024-04-27T07:50:50.923685+0000","last_clean":"2024-04-27T07:50:50.923685+0000","last_became_active":"2024-04-27T07:50:48.803586+0000","last_became_peered":"2024-04-27T07:50:48.803586+0000","last_unstale":"2024-04-27T07:50:50.923685+0000","last_undegraded":"2024-04-27T07:50:50.923685+0000","last_fullsized":"2024-04-27T07:50:50.923685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282373+0000","last_change":"2024-04-27T07:50:47.796755+0000","last_active":"2024-04-27T07:50:51.282373+0000","last_peered":"2024-04-27T07:50:51.282373+0000","last_clean":"2024-04-27T07:50:51.282373+0000","last_became_active":"2024-04-27T07:50:47.796096+0000","last_became_peered":"2024-04-27T07:50:47.796096+0000","last_unstale":"2024-04-27T07:50:51.282373+0000","last_undegraded":"2024-04-27T07:50:51.282373+0000","last_fullsized":"2024-04-27T07:50:51.282373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799129+0000","last_change":"2024-04-27T07:50:48.792778+0000","last_active":"2024-04-27T07:50:49.799129+0000","last_peered":"2024-04-27T07:50:49.799129+0000","last_clean":"2024-04-27T07:50:49.799129+0000","last_became_active":"2024-04-27T07:50:48.791993+0000","last_became_peered":"2024-04-27T07:50:48.791993+0000","last_unstale":"2024-04-27T07:50:49.799129+0000","last_undegraded":"2024-04-27T07:50:49.799129+0000","last_fullsized":"2024-04-27T07:50:49.799129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:02.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281677+0000","last_change":"2024-04-27T07:50:47.785296+0000","last_active":"2024-04-27T07:50:51.281677+0000","last_peered":"2024-04-27T07:50:51.281677+0000","last_clean":"2024-04-27T07:50:51.281677+0000","last_became_active":"2024-04-27T07:50:47.784874+0000","last_became_peered":"2024-04-27T07:50:47.784874+0000","last_unstale":"2024-04-27T07:50:51.281677+0000","last_undegraded":"2024-04-27T07:50:51.281677+0000","last_fullsized":"2024-04-27T07:50:51.281677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281489+0000","last_change":"2024-04-27T07:50:48.809822+0000","last_active":"2024-04-27T07:50:51.281489+0000","last_peered":"2024-04-27T07:50:51.281489+0000","last_clean":"2024-04-27T07:50:51.281489+0000","last_became_active":"2024-04-27T07:50:48.809495+0000","last_became_peered":"2024-04-27T07:50:48.809495+0000","last_unstale":"2024-04-27T07:50:51.281489+0000","last_undegraded":"2024-04-27T07:50:51.281489+0000","last_fullsized":"2024-04-27T07:50:51.281489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281879+0000","last_change":"2024-04-27T07:50:47.777251+0000","last_active":"2024-04-27T07:50:51.281879+0000","last_peered":"2024-04-27T07:50:51.281879+0000","last_clean":"2024-04-27T07:50:51.281879+0000","last_became_active":"2024-04-27T07:50:47.776893+0000","last_became_peered":"2024-04-27T07:50:47.776893+0000","last_unstale":"2024-04-27T07:50:51.281879+0000","last_undegraded":"2024-04-27T07:50:51.281879+0000","last_fullsized":"2024-04-27T07:50:51.281879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282509+0000","last_change":"2024-04-27T07:50:48.821019+0000","last_active":"2024-04-27T07:50:51.282509+0000","last_peered":"2024-04-27T07:50:51.282509+0000","last_clean":"2024-04-27T07:50:51.282509+0000","last_became_active":"2024-04-27T07:50:48.820606+0000","last_became_peered":"2024-04-27T07:50:48.820606+0000","last_unstale":"2024-04-27T07:50:51.282509+0000","last_undegraded":"2024-04-27T07:50:51.282509+0000","last_fullsized":"2024-04-27T07:50:51.282509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283602+0000","last_change":"2024-04-27T07:50:47.800388+0000","last_active":"2024-04-27T07:50:51.283602+0000","last_peered":"2024-04-27T07:50:51.283602+0000","last_clean":"2024-04-27T07:50:51.283602+0000","last_became_active":"2024-04-27T07:50:47.799604+0000","last_became_peered":"2024-04-27T07:50:47.799604+0000","last_unstale":"2024-04-27T07:50:51.283602+0000","last_undegraded":"2024-04-27T07:50:51.283602+0000","last_fullsized":"2024-04-27T07:50:51.283602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:34:31.234332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445740+0000","last_change":"2024-04-27T07:50:48.810527+0000","last_active":"2024-04-27T07:50:50.445740+0000","last_peered":"2024-04-27T07:50:50.445740+0000","last_clean":"2024-04-27T07:50:50.445740+0000","last_became_active":"2024-04-27T07:50:48.810203+0000","last_became_peered":"2024-04-27T07:50:48.810203+0000","last_unstale":"2024-04-27T07:50:50.445740+0000","last_undegraded":"2024-04-27T07:50:50.445740+0000","last_fullsized":"2024-04-27T07:50:50.445740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:59.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799691+0000","last_change":"2024-04-27T07:50:48.787926+0000","last_active":"2024-04-27T07:50:49.799691+0000","last_peered":"2024-04-27T07:50:49.799691+0000","last_clean":"2024-04-27T07:50:49.799691+0000","last_became_active":"2024-04-27T07:50:48.787213+0000","last_became_peered":"2024-04-27T07:50:48.787213+0000","last_unstale":"2024-04-27T07:50:49.799691+0000","last_undegraded":"2024-04-27T07:50:49.799691+0000","last_fullsized":"2024-04-27T07:50:49.799691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282943+0000","last_change":"2024-04-27T07:50:47.799584+0000","last_active":"2024-04-27T07:50:51.282943+0000","last_peered":"2024-04-27T07:50:51.282943+0000","last_clean":"2024-04-27T07:50:51.282943+0000","last_became_active":"2024-04-27T07:50:47.799182+0000","last_became_peered":"2024-04-27T07:50:47.799182+0000","last_unstale":"2024-04-27T07:50:51.282943+0000","last_undegraded":"2024-04-27T07:50:51.282943+0000","last_fullsized":"2024-04-27T07:50:51.282943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:31:29.207109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801755+0000","last_change":"2024-04-27T07:50:48.798910+0000","last_active":"2024-04-27T07:50:49.801755+0000","last_peered":"2024-04-27T07:50:49.801755+0000","last_clean":"2024-04-27T07:50:49.801755+0000","last_became_active":"2024-04-27T07:50:48.798376+0000","last_became_peered":"2024-04-27T07:50:48.798376+0000","last_unstale":"2024-04-27T07:50:49.801755+0000","last_undegraded":"2024-04-27T07:50:49.801755+0000","last_fullsized":"2024-04-27T07:50:49.801755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:17:52.504141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283271+0000","last_change":"2024-04-27T07:50:47.798503+0000","last_active":"2024-04-27T07:50:51.283271+0000","last_peered":"2024-04-27T07:50:51.283271+0000","last_clean":"2024-04-27T07:50:51.283271+0000","last_became_active":"2024-04-27T07:50:47.798171+0000","last_became_peered":"2024-04-27T07:50:47.798171+0000","last_unstale":"2024-04-27T07:50:51.283271+0000","last_undegraded":"2024-04-27T07:50:51.283271+0000","last_fullsized":"2024-04-27T07:50:51.283271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:16.910153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923642+0000","last_change":"2024-04-27T07:50:48.818082+0000","last_active":"2024-04-27T07:50:50.923642+0000","last_peered":"2024-04-27T07:50:50.923642+0000","last_clean":"2024-04-27T07:50:50.923642+0000","last_became_active":"2024-04-27T07:50:48.813287+0000","last_became_peered":"2024-04-27T07:50:48.813287+0000","last_unstale":"2024-04-27T07:50:50.923642+0000","last_undegraded":"2024-04-27T07:50:50.923642+0000","last_fullsized":"2024-04-27T07:50:50.923642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282997+0000","last_change":"2024-04-27T07:50:47.770647+0000","last_active":"2024-04-27T07:50:51.282997+0000","last_peered":"2024-04-27T07:50:51.282997+0000","last_clean":"2024-04-27T07:50:51.282997+0000","last_became_active":"2024-04-27T07:50:47.770223+0000","last_became_peered":"2024-04-27T07:50:47.770223+0000","last_unstale":"2024-04-27T07:50:51.282997+0000","last_undegraded":"2024-04-27T07:50:51.282997+0000","last_fullsized":"2024-04-27T07:50:51.282997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282890+0000","last_change":"2024-04-27T07:50:48.828744+0000","last_active":"2024-04-27T07:50:51.282890+0000","last_peered":"2024-04-27T07:50:51.282890+0000","last_clean":"2024-04-27T07:50:51.282890+0000","last_became_active":"2024-04-27T07:50:48.810916+0000","last_became_peered":"2024-04-27T07:50:48.810916+0000","last_unstale":"2024-04-27T07:50:51.282890+0000","last_undegraded":"2024-04-27T07:50:51.282890+0000","last_fullsized":"2024-04-27T07:50:51.282890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923774+0000","last_change":"2024-04-27T07:50:47.785683+0000","last_active":"2024-04-27T07:50:50.923774+0000","last_peered":"2024-04-27T07:50:50.923774+0000","last_clean":"2024-04-27T07:50:50.923774+0000","last_became_active":"2024-04-27T07:50:47.784993+0000","last_became_peered":"2024-04-27T07:50:47.784993+0000","last_unstale":"2024-04-27T07:50:50.923774+0000","last_undegraded":"2024-04-27T07:50:50.923774+0000","last_fullsized":"2024-04-27T07:50:50.923774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:49.046466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:51:01.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:51:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.039+0000 7fbc2cc90640 1 Processor -- start 2024-04-27T07:51:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.039+0000 7fbc2cc90640 1 -- start start 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2cc90640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc28169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2cc90640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc280cff70 con 0x7fbc280d4790 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2cc90640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc280b3d30 con 0x7fbc2815ebe0 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2cc90640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbc280d4540 con 0x7fbc280b7930 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2659c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbc280d4790 0x7fbc280d4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41234/0 (socket says 172.21.15.22:41234) 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2659c640 1 -- 172.21.15.22:0/1007040956 learned_addr learned my addr 172.21.15.22:0/1007040956 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc26d9d640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc28169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261890277 0 0) 0x7fbc280cff70 con 0x7fbc280d4790 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc10003530 con 0x7fbc280d4790 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1909199865 0 0) 0x7fbc10003530 con 0x7fbc280d4790 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 msgr2=0x7fbc28169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc28169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 >> v1:172.21.15.183:6789/0 conn(0x7fbc2815ebe0 legacy=0x7fbc280cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc280ba7a0 con 0x7fbc280d4790 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fbc1c001680 con 0x7fbc280d4790 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc26d9d640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc28169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 >> v1:172.21.15.22:6789/0 conn(0x7fbc280d4790 legacy=0x7fbc280d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc10004d50 0x7fbc100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280d4540 con 0x7fbc10003fe0 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1000bbc0 con 0x7fbc10004d50 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1000bdc0 con 0x7fbc280b7930 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc26d9d640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc10004d50 0x7fbc100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc2659c640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 -- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 msgr2=0x7fbc100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc280b7930 0x7fbc100039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 -- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc10004d50 msgr2=0x7fbc100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc10004d50 0x7fbc100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.043+0000 7fbc25d9b640 1 -- 172.21.15.22:0/1007040956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc10003530 con 0x7fbc10003fe0 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc18001750 tx=0x7fbc1801c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7f35154af32cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc180013a0 con 0x7fbc10003fe0 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc180052f0 con 0x7fbc10003fe0 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2559a640 1 -- 172.21.15.22:0/1007040956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18003840 con 0x7fbc10003fe0 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/1007040956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc10003fe0 msgr2=0x7fbc10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/1007040956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc18001750 tx=0x7fbc1801c200 comp rx=0 tx=0).stop 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc26d9d640 1 -- 172.21.15.22:0/1007040956 reap_dead start 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/1007040956 shutdown_connections 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/1007040956 >> 172.21.15.22:0/1007040956 conn(0x7fbc28067d10 msgr2=0x7fbc28068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/1007040956 shutdown_connections 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/1007040956 wait complete. 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 Processor -- start 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- start start 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc10003fe0 0x7fbc281f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc28159c70 0x7fbc2815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280ba2a0 con 0x7fbc280b6e80 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280ba120 con 0x7fbc28159c70 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbc28169d20 con 0x7fbc10003fe0 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40832/0 (socket says 172.21.15.22:40832) 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 -- 172.21.15.22:0/3196190860 learned_addr learned my addr 172.21.15.22:0/3196190860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc26d9d640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc28159c70 0x7fbc2815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2659c640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc10003fe0 0x7fbc281f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 -- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc10003fe0 msgr2=0x7fbc281f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc10003fe0 0x7fbc281f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 -- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc28159c70 msgr2=0x7fbc2815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc28159c70 0x7fbc2815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 -- 172.21.15.22:0/3196190860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc28161700 con 0x7fbc280b6e80 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc25d9b640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc1801c6e0 tx=0x7fbc10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18007a40 con 0x7fbc280b6e80 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc180bb070 con 0x7fbc280b6e80 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc281619e0 con 0x7fbc280b6e80 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc180013a0 con 0x7fbc280b6e80 2024-04-27T07:51:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.047+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc280b45d0 con 0x7fbc280b6e80 2024-04-27T07:51:02.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc28068c30 con 0x7fbc280b6e80 2024-04-27T07:51:02.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbc180b4040 con 0x7fbc280b6e80 2024-04-27T07:51:02.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc16ffd640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 0x7fbc0806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc181208a0 con 0x7fbc280b6e80 2024-04-27T07:51:02.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc2659c640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 0x7fbc0806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.051+0000 7fbc2659c640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 0x7fbc0806a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0059c0 tx=0x7fbc1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.055+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbc280b3d30 con 0x7fbc280b6e80 2024-04-27T07:51:02.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.207+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc280b99f0 con 0x7fbc080683b0 2024-04-27T07:51:02.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.227+0000 7fbc16ffd640 1 -- 172.21.15.22:0/3196190860 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fbc280b99f0 con 0x7fbc080683b0 2024-04-27T07:51:02.234 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.227+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 msgr2=0x7fbc0806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 0x7fbc0806a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0059c0 tx=0x7fbc1c009040 comp rx=0 tx=0).stop 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 msgr2=0x7fbc2815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc1801c6e0 tx=0x7fbc10005840 comp rx=0 tx=0).stop 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 shutdown_connections 2024-04-27T07:51:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fbc080683b0 0x7fbc0806a870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc28159c70 0x7fbc2815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc280b6e80 0x7fbc2815efd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 --2- 172.21.15.22:0/3196190860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc10003fe0 0x7fbc281f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 >> 172.21.15.22:0/3196190860 conn(0x7fbc28067d10 msgr2=0x7fbc280b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 shutdown_connections 2024-04-27T07:51:02.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.231+0000 7fbc2cc90640 1 -- 172.21.15.22:0/3196190860 wait complete. 2024-04-27T07:51:02.250 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-27T07:51:02.152799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235696,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739024,"statfs":{"total":773094113280,"available":772852760576,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004593"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283542+0000","last_change":"2024-04-27T07:50:48.807748+0000","last_active":"2024-04-27T07:50:51.283542+0000","last_peered":"2024-04-27T07:50:51.283542+0000","last_clean":"2024-04-27T07:50:51.283542+0000","last_became_active":"2024-04-27T07:50:48.806582+0000","last_became_peered":"2024-04-27T07:50:48.806582+0000","last_unstale":"2024-04-27T07:50:51.283542+0000","last_undegraded":"2024-04-27T07:50:51.283542+0000","last_fullsized":"2024-04-27T07:50:51.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.223671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282205+0000","last_change":"2024-04-27T07:50:48.809136+0000","last_active":"2024-04-27T07:50:51.282205+0000","last_peered":"2024-04-27T07:50:51.282205+0000","last_clean":"2024-04-27T07:50:51.282205+0000","last_became_active":"2024-04-27T07:50:48.808581+0000","last_became_peered":"2024-04-27T07:50:48.808581+0000","last_unstale":"2024-04-27T07:50:51.282205+0000","last_undegraded":"2024-04-27T07:50:51.282205+0000","last_fullsized":"2024-04-27T07:50:51.282205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923394+0000","last_change":"2024-04-27T07:50:48.803232+0000","last_active":"2024-04-27T07:50:50.923394+0000","last_peered":"2024-04-27T07:50:50.923394+0000","last_clean":"2024-04-27T07:50:50.923394+0000","last_became_active":"2024-04-27T07:50:48.802828+0000","last_became_peered":"2024-04-27T07:50:48.802828+0000","last_unstale":"2024-04-27T07:50:50.923394+0000","last_undegraded":"2024-04-27T07:50:50.923394+0000","last_fullsized":"2024-04-27T07:50:50.923394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800701+0000","last_change":"2024-04-27T07:50:48.822942+0000","last_active":"2024-04-27T07:50:49.800701+0000","last_peered":"2024-04-27T07:50:49.800701+0000","last_clean":"2024-04-27T07:50:49.800701+0000","last_became_active":"2024-04-27T07:50:48.822709+0000","last_became_peered":"2024-04-27T07:50:48.822709+0000","last_unstale":"2024-04-27T07:50:49.800701+0000","last_undegraded":"2024-04-27T07:50:49.800701+0000","last_fullsized":"2024-04-27T07:50:49.800701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:38.768492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283631+0000","last_change":"2024-04-27T07:50:48.805852+0000","last_active":"2024-04-27T07:50:51.283631+0000","last_peered":"2024-04-27T07:50:51.283631+0000","last_clean":"2024-04-27T07:50:51.283631+0000","last_became_active":"2024-04-27T07:50:48.805553+0000","last_became_peered":"2024-04-27T07:50:48.805553+0000","last_unstale":"2024-04-27T07:50:51.283631+0000","last_undegraded":"2024-04-27T07:50:51.283631+0000","last_fullsized":"2024-04-27T07:50:51.283631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:19:06.682886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800634+0000","last_change":"2024-04-27T07:50:48.795482+0000","last_active":"2024-04-27T07:50:49.800634+0000","last_peered":"2024-04-27T07:50:49.800634+0000","last_clean":"2024-04-27T07:50:49.800634+0000","last_became_active":"2024-04-27T07:50:48.793813+0000","last_became_peered":"2024-04-27T07:50:48.793813+0000","last_unstale":"2024-04-27T07:50:49.800634+0000","last_undegraded":"2024-04-27T07:50:49.800634+0000","last_fullsized":"2024-04-27T07:50:49.800634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:39:48.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282786+0000","last_change":"2024-04-27T07:50:48.821409+0000","last_active":"2024-04-27T07:50:51.282786+0000","last_peered":"2024-04-27T07:50:51.282786+0000","last_clean":"2024-04-27T07:50:51.282786+0000","last_became_active":"2024-04-27T07:50:48.821162+0000","last_became_peered":"2024-04-27T07:50:48.821162+0000","last_unstale":"2024-04-27T07:50:51.282786+0000","last_undegraded":"2024-04-27T07:50:51.282786+0000","last_fullsized":"2024-04-27T07:50:51.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:20:43.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281085+0000","last_change":"2024-04-27T07:50:48.830026+0000","last_active":"2024-04-27T07:50:51.281085+0000","last_peered":"2024-04-27T07:50:51.281085+0000","last_clean":"2024-04-27T07:50:51.281085+0000","last_became_active":"2024-04-27T07:50:48.821303+0000","last_became_peered":"2024-04-27T07:50:48.821303+0000","last_unstale":"2024-04-27T07:50:51.281085+0000","last_undegraded":"2024-04-27T07:50:51.281085+0000","last_fullsized":"2024-04-27T07:50:51.281085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283709+0000","last_change":"2024-04-27T07:50:48.820925+0000","last_active":"2024-04-27T07:50:51.283709+0000","last_peered":"2024-04-27T07:50:51.283709+0000","last_clean":"2024-04-27T07:50:51.283709+0000","last_became_active":"2024-04-27T07:50:48.820575+0000","last_became_peered":"2024-04-27T07:50:48.820575+0000","last_unstale":"2024-04-27T07:50:51.283709+0000","last_undegraded":"2024-04-27T07:50:51.283709+0000","last_fullsized":"2024-04-27T07:50:51.283709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:40.499571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446110+0000","last_change":"2024-04-27T07:50:48.814093+0000","last_active":"2024-04-27T07:50:50.446110+0000","last_peered":"2024-04-27T07:50:50.446110+0000","last_clean":"2024-04-27T07:50:50.446110+0000","last_became_active":"2024-04-27T07:50:48.813643+0000","last_became_peered":"2024-04-27T07:50:48.813643+0000","last_unstale":"2024-04-27T07:50:50.446110+0000","last_undegraded":"2024-04-27T07:50:50.446110+0000","last_fullsized":"2024-04-27T07:50:50.446110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:24.403447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445905+0000","last_change":"2024-04-27T07:50:48.815381+0000","last_active":"2024-04-27T07:50:50.445905+0000","last_peered":"2024-04-27T07:50:50.445905+0000","last_clean":"2024-04-27T07:50:50.445905+0000","last_became_active":"2024-04-27T07:50:48.814574+0000","last_became_peered":"2024-04-27T07:50:48.814574+0000","last_unstale":"2024-04-27T07:50:50.445905+0000","last_undegraded":"2024-04-27T07:50:50.445905+0000","last_fullsized":"2024-04-27T07:50:50.445905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:25.162778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800227+0000","last_change":"2024-04-27T07:50:48.806044+0000","last_active":"2024-04-27T07:50:49.800227+0000","last_peered":"2024-04-27T07:50:49.800227+0000","last_clean":"2024-04-27T07:50:49.800227+0000","last_became_active":"2024-04-27T07:50:48.805705+0000","last_became_peered":"2024-04-27T07:50:48.805705+0000","last_unstale":"2024-04-27T07:50:49.800227+0000","last_undegraded":"2024-04-27T07:50:49.800227+0000","last_fullsized":"2024-04-27T07:50:49.800227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:21.183002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282310+0000","last_change":"2024-04-27T07:50:48.819990+0000","last_active":"2024-04-27T07:50:51.282310+0000","last_peered":"2024-04-27T07:50:51.282310+0000","last_clean":"2024-04-27T07:50:51.282310+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.282310+0000","last_undegraded":"2024-04-27T07:50:51.282310+0000","last_fullsized":"2024-04-27T07:50:51.282310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922720+0000","last_change":"2024-04-27T07:50:48.804618+0000","last_active":"2024-04-27T07:50:50.922720+0000","last_peered":"2024-04-27T07:50:50.922720+0000","last_clean":"2024-04-27T07:50:50.922720+0000","last_became_active":"2024-04-27T07:50:48.803939+0000","last_became_peered":"2024-04-27T07:50:48.803939+0000","last_unstale":"2024-04-27T07:50:50.922720+0000","last_undegraded":"2024-04-27T07:50:50.922720+0000","last_fullsized":"2024-04-27T07:50:50.922720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:38.135106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281763+0000","last_change":"2024-04-27T07:50:48.813856+0000","last_active":"2024-04-27T07:50:51.281763+0000","last_peered":"2024-04-27T07:50:51.281763+0000","last_clean":"2024-04-27T07:50:51.281763+0000","last_became_active":"2024-04-27T07:50:48.813437+0000","last_became_peered":"2024-04-27T07:50:48.813437+0000","last_unstale":"2024-04-27T07:50:51.281763+0000","last_undegraded":"2024-04-27T07:50:51.281763+0000","last_fullsized":"2024-04-27T07:50:51.281763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801741+0000","last_change":"2024-04-27T07:50:48.810807+0000","last_active":"2024-04-27T07:50:49.801741+0000","last_peered":"2024-04-27T07:50:49.801741+0000","last_clean":"2024-04-27T07:50:49.801741+0000","last_became_active":"2024-04-27T07:50:48.810545+0000","last_became_peered":"2024-04-27T07:50:48.810545+0000","last_unstale":"2024-04-27T07:50:49.801741+0000","last_undegraded":"2024-04-27T07:50:49.801741+0000","last_fullsized":"2024-04-27T07:50:49.801741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:09.348372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799832+0000","last_change":"2024-04-27T07:50:48.801260+0000","last_active":"2024-04-27T07:50:49.799832+0000","last_peered":"2024-04-27T07:50:49.799832+0000","last_clean":"2024-04-27T07:50:49.799832+0000","last_became_active":"2024-04-27T07:50:48.800565+0000","last_became_peered":"2024-04-27T07:50:48.800565+0000","last_unstale":"2024-04-27T07:50:49.799832+0000","last_undegraded":"2024-04-27T07:50:49.799832+0000","last_fullsized":"2024-04-27T07:50:49.799832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797791+0000","last_change":"2024-04-27T07:50:48.806722+0000","last_active":"2024-04-27T07:50:49.797791+0000","last_peered":"2024-04-27T07:50:49.797791+0000","last_clean":"2024-04-27T07:50:49.797791+0000","last_became_active":"2024-04-27T07:50:48.805899+0000","last_became_peered":"2024-04-27T07:50:48.805899+0000","last_unstale":"2024-04-27T07:50:49.797791+0000","last_undegraded":"2024-04-27T07:50:49.797791+0000","last_fullsized":"2024-04-27T07:50:49.797791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:28:00.765509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283852+0000","last_change":"2024-04-27T07:50:48.806262+0000","last_active":"2024-04-27T07:50:51.283852+0000","last_peered":"2024-04-27T07:50:51.283852+0000","last_clean":"2024-04-27T07:50:51.283852+0000","last_became_active":"2024-04-27T07:50:48.805987+0000","last_became_peered":"2024-04-27T07:50:48.805987+0000","last_unstale":"2024-04-27T07:50:51.283852+0000","last_undegraded":"2024-04-27T07:50:51.283852+0000","last_fullsized":"2024-04-27T07:50:51.283852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:49:40.263257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445516+0000","last_change":"2024-04-27T07:50:48.811277+0000","last_active":"2024-04-27T07:50:50.445516+0000","last_peered":"2024-04-27T07:50:50.445516+0000","last_clean":"2024-04-27T07:50:50.445516+0000","last_became_active":"2024-04-27T07:50:48.810296+0000","last_became_peered":"2024-04-27T07:50:48.810296+0000","last_unstale":"2024-04-27T07:50:50.445516+0000","last_undegraded":"2024-04-27T07:50:50.445516+0000","last_fullsized":"2024-04-27T07:50:50.445516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:43:55.556626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922970+0000","last_change":"2024-04-27T07:50:48.798022+0000","last_active":"2024-04-27T07:50:50.922970+0000","last_peered":"2024-04-27T07:50:50.922970+0000","last_clean":"2024-04-27T07:50:50.922970+0000","last_became_active":"2024-04-27T07:50:48.797464+0000","last_became_peered":"2024-04-27T07:50:48.797464+0000","last_unstale":"2024-04-27T07:50:50.922970+0000","last_undegraded":"2024-04-27T07:50:50.922970+0000","last_fullsized":"2024-04-27T07:50:50.922970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:39:13.996688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800758+0000","last_change":"2024-04-27T07:50:48.796045+0000","last_active":"2024-04-27T07:50:49.800758+0000","last_peered":"2024-04-27T07:50:49.800758+0000","last_clean":"2024-04-27T07:50:49.800758+0000","last_became_active":"2024-04-27T07:50:48.794825+0000","last_became_peered":"2024-04-27T07:50:48.794825+0000","last_unstale":"2024-04-27T07:50:49.800758+0000","last_undegraded":"2024-04-27T07:50:49.800758+0000","last_fullsized":"2024-04-27T07:50:49.800758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:58:49.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281332+0000","last_change":"2024-04-27T07:50:48.821433+0000","last_active":"2024-04-27T07:50:51.281332+0000","last_peered":"2024-04-27T07:50:51.281332+0000","last_clean":"2024-04-27T07:50:51.281332+0000","last_became_active":"2024-04-27T07:50:48.820660+0000","last_became_peered":"2024-04-27T07:50:48.820660+0000","last_unstale":"2024-04-27T07:50:51.281332+0000","last_undegraded":"2024-04-27T07:50:51.281332+0000","last_fullsized":"2024-04-27T07:50:51.281332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799368+0000","last_change":"2024-04-27T07:50:48.810201+0000","last_active":"2024-04-27T07:50:49.799368+0000","last_peered":"2024-04-27T07:50:49.799368+0000","last_clean":"2024-04-27T07:50:49.799368+0000","last_became_active":"2024-04-27T07:50:48.809937+0000","last_became_peered":"2024-04-27T07:50:48.809937+0000","last_unstale":"2024-04-27T07:50:49.799368+0000","last_undegraded":"2024-04-27T07:50:49.799368+0000","last_fullsized":"2024-04-27T07:50:49.799368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283175+0000","last_change":"2024-04-27T07:50:48.810567+0000","last_active":"2024-04-27T07:50:51.283175+0000","last_peered":"2024-04-27T07:50:51.283175+0000","last_clean":"2024-04-27T07:50:51.283175+0000","last_became_active":"2024-04-27T07:50:48.809921+0000","last_became_peered":"2024-04-27T07:50:48.809921+0000","last_unstale":"2024-04-27T07:50:51.283175+0000","last_undegraded":"2024-04-27T07:50:51.283175+0000","last_fullsized":"2024-04-27T07:50:51.283175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:50.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.446226+0000","last_change":"2024-04-27T07:50:48.805860+0000","last_active":"2024-04-27T07:50:50.446226+0000","last_peered":"2024-04-27T07:50:50.446226+0000","last_clean":"2024-04-27T07:50:50.446226+0000","last_became_active":"2024-04-27T07:50:48.805182+0000","last_became_peered":"2024-04-27T07:50:48.805182+0000","last_unstale":"2024-04-27T07:50:50.446226+0000","last_undegraded":"2024-04-27T07:50:50.446226+0000","last_fullsized":"2024-04-27T07:50:50.446226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:10:53.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281842+0000","last_change":"2024-04-27T07:50:48.811553+0000","last_active":"2024-04-27T07:50:51.281842+0000","last_peered":"2024-04-27T07:50:51.281842+0000","last_clean":"2024-04-27T07:50:51.281842+0000","last_became_active":"2024-04-27T07:50:48.810935+0000","last_became_peered":"2024-04-27T07:50:48.810935+0000","last_unstale":"2024-04-27T07:50:51.281842+0000","last_undegraded":"2024-04-27T07:50:51.281842+0000","last_fullsized":"2024-04-27T07:50:51.281842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283139+0000","last_change":"2024-04-27T07:50:48.820547+0000","last_active":"2024-04-27T07:50:51.283139+0000","last_peered":"2024-04-27T07:50:51.283139+0000","last_clean":"2024-04-27T07:50:51.283139+0000","last_became_active":"2024-04-27T07:50:48.820170+0000","last_became_peered":"2024-04-27T07:50:48.820170+0000","last_unstale":"2024-04-27T07:50:51.283139+0000","last_undegraded":"2024-04-27T07:50:51.283139+0000","last_fullsized":"2024-04-27T07:50:51.283139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923336+0000","last_change":"2024-04-27T07:50:48.821676+0000","last_active":"2024-04-27T07:50:50.923336+0000","last_peered":"2024-04-27T07:50:50.923336+0000","last_clean":"2024-04-27T07:50:50.923336+0000","last_became_active":"2024-04-27T07:50:48.820884+0000","last_became_peered":"2024-04-27T07:50:48.820884+0000","last_unstale":"2024-04-27T07:50:50.923336+0000","last_undegraded":"2024-04-27T07:50:50.923336+0000","last_fullsized":"2024-04-27T07:50:50.923336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800884+0000","last_change":"2024-04-27T07:50:48.808706+0000","last_active":"2024-04-27T07:50:49.800884+0000","last_peered":"2024-04-27T07:50:49.800884+0000","last_clean":"2024-04-27T07:50:49.800884+0000","last_became_active":"2024-04-27T07:50:48.804475+0000","last_became_peered":"2024-04-27T07:50:48.804475+0000","last_unstale":"2024-04-27T07:50:49.800884+0000","last_undegraded":"2024-04-27T07:50:49.800884+0000","last_fullsized":"2024-04-27T07:50:49.800884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:07:50.589385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797415+0000","last_change":"2024-04-27T07:50:48.810016+0000","last_active":"2024-04-27T07:50:49.797415+0000","last_peered":"2024-04-27T07:50:49.797415+0000","last_clean":"2024-04-27T07:50:49.797415+0000","last_became_active":"2024-04-27T07:50:48.809618+0000","last_became_peered":"2024-04-27T07:50:48.809618+0000","last_unstale":"2024-04-27T07:50:49.797415+0000","last_undegraded":"2024-04-27T07:50:49.797415+0000","last_fullsized":"2024-04-27T07:50:49.797415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:20.530307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445841+0000","last_change":"2024-04-27T07:50:48.805951+0000","last_active":"2024-04-27T07:50:50.445841+0000","last_peered":"2024-04-27T07:50:50.445841+0000","last_clean":"2024-04-27T07:50:50.445841+0000","last_became_active":"2024-04-27T07:50:48.805493+0000","last_became_peered":"2024-04-27T07:50:48.805493+0000","last_unstale":"2024-04-27T07:50:50.445841+0000","last_undegraded":"2024-04-27T07:50:50.445841+0000","last_fullsized":"2024-04-27T07:50:50.445841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:42:36.766605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800350+0000","last_change":"2024-04-27T07:50:48.799052+0000","last_active":"2024-04-27T07:50:49.800350+0000","last_peered":"2024-04-27T07:50:49.800350+0000","last_clean":"2024-04-27T07:50:49.800350+0000","last_became_active":"2024-04-27T07:50:48.798769+0000","last_became_peered":"2024-04-27T07:50:48.798769+0000","last_unstale":"2024-04-27T07:50:49.800350+0000","last_undegraded":"2024-04-27T07:50:49.800350+0000","last_fullsized":"2024-04-27T07:50:49.800350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:19.179469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282661+0000","last_change":"2024-04-27T07:50:47.787481+0000","last_active":"2024-04-27T07:50:51.282661+0000","last_peered":"2024-04-27T07:50:51.282661+0000","last_clean":"2024-04-27T07:50:51.282661+0000","last_became_active":"2024-04-27T07:50:47.787133+0000","last_became_peered":"2024-04-27T07:50:47.787133+0000","last_unstale":"2024-04-27T07:50:51.282661+0000","last_undegraded":"2024-04-27T07:50:51.282661+0000","last_fullsized":"2024-04-27T07:50:51.282661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:50.164872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923143+0000","last_change":"2024-04-27T07:50:48.812526+0000","last_active":"2024-04-27T07:50:50.923143+0000","last_peered":"2024-04-27T07:50:50.923143+0000","last_clean":"2024-04-27T07:50:50.923143+0000","last_became_active":"2024-04-27T07:50:48.812099+0000","last_became_peered":"2024-04-27T07:50:48.812099+0000","last_unstale":"2024-04-27T07:50:50.923143+0000","last_undegraded":"2024-04-27T07:50:50.923143+0000","last_fullsized":"2024-04-27T07:50:50.923143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445014+0000","last_change":"2024-04-27T07:50:47.786268+0000","last_active":"2024-04-27T07:50:50.445014+0000","last_peered":"2024-04-27T07:50:50.445014+0000","last_clean":"2024-04-27T07:50:50.445014+0000","last_became_active":"2024-04-27T07:50:47.785467+0000","last_became_peered":"2024-04-27T07:50:47.785467+0000","last_unstale":"2024-04-27T07:50:50.445014+0000","last_undegraded":"2024-04-27T07:50:50.445014+0000","last_fullsized":"2024-04-27T07:50:50.445014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283343+0000","last_change":"2024-04-27T07:50:47.782977+0000","last_active":"2024-04-27T07:50:51.283343+0000","last_peered":"2024-04-27T07:50:51.283343+0000","last_clean":"2024-04-27T07:50:51.283343+0000","last_became_active":"2024-04-27T07:50:47.782680+0000","last_became_peered":"2024-04-27T07:50:47.782680+0000","last_unstale":"2024-04-27T07:50:51.283343+0000","last_undegraded":"2024-04-27T07:50:51.283343+0000","last_fullsized":"2024-04-27T07:50:51.283343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281283+0000","last_change":"2024-04-27T07:50:48.795764+0000","last_active":"2024-04-27T07:50:51.281283+0000","last_peered":"2024-04-27T07:50:51.281283+0000","last_clean":"2024-04-27T07:50:51.281283+0000","last_became_active":"2024-04-27T07:50:48.794946+0000","last_became_peered":"2024-04-27T07:50:48.794946+0000","last_unstale":"2024-04-27T07:50:51.281283+0000","last_undegraded":"2024-04-27T07:50:51.281283+0000","last_fullsized":"2024-04-27T07:50:51.281283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799501+0000","last_change":"2024-04-27T07:50:47.768634+0000","last_active":"2024-04-27T07:50:49.799501+0000","last_peered":"2024-04-27T07:50:49.799501+0000","last_clean":"2024-04-27T07:50:49.799501+0000","last_became_active":"2024-04-27T07:50:47.767344+0000","last_became_peered":"2024-04-27T07:50:47.767344+0000","last_unstale":"2024-04-27T07:50:49.799501+0000","last_undegraded":"2024-04-27T07:50:49.799501+0000","last_fullsized":"2024-04-27T07:50:49.799501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281548+0000","last_change":"2024-04-27T07:50:48.831910+0000","last_active":"2024-04-27T07:50:51.281548+0000","last_peered":"2024-04-27T07:50:51.281548+0000","last_clean":"2024-04-27T07:50:51.281548+0000","last_became_active":"2024-04-27T07:50:48.831339+0000","last_became_peered":"2024-04-27T07:50:48.831339+0000","last_unstale":"2024-04-27T07:50:51.281548+0000","last_undegraded":"2024-04-27T07:50:51.281548+0000","last_fullsized":"2024-04-27T07:50:51.281548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:18.739183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445402+0000","last_change":"2024-04-27T07:50:47.784442+0000","last_active":"2024-04-27T07:50:50.445402+0000","last_peered":"2024-04-27T07:50:50.445402+0000","last_clean":"2024-04-27T07:50:50.445402+0000","last_became_active":"2024-04-27T07:50:47.783769+0000","last_became_peered":"2024-04-27T07:50:47.783769+0000","last_unstale":"2024-04-27T07:50:50.445402+0000","last_undegraded":"2024-04-27T07:50:50.445402+0000","last_fullsized":"2024-04-27T07:50:50.445402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:07:15.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799300+0000","last_change":"2024-04-27T07:50:48.806497+0000","last_active":"2024-04-27T07:50:49.799300+0000","last_peered":"2024-04-27T07:50:49.799300+0000","last_clean":"2024-04-27T07:50:49.799300+0000","last_became_active":"2024-04-27T07:50:48.806172+0000","last_became_peered":"2024-04-27T07:50:48.806172+0000","last_unstale":"2024-04-27T07:50:49.799300+0000","last_undegraded":"2024-04-27T07:50:49.799300+0000","last_fullsized":"2024-04-27T07:50:49.799300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799348+0000","last_change":"2024-04-27T07:50:47.776964+0000","last_active":"2024-04-27T07:50:49.799348+0000","last_peered":"2024-04-27T07:50:49.799348+0000","last_clean":"2024-04-27T07:50:49.799348+0000","last_became_active":"2024-04-27T07:50:47.776179+0000","last_became_peered":"2024-04-27T07:50:47.776179+0000","last_unstale":"2024-04-27T07:50:49.799348+0000","last_undegraded":"2024-04-27T07:50:49.799348+0000","last_fullsized":"2024-04-27T07:50:49.799348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923673+0000","last_change":"2024-04-27T07:50:48.812739+0000","last_active":"2024-04-27T07:50:50.923673+0000","last_peered":"2024-04-27T07:50:50.923673+0000","last_clean":"2024-04-27T07:50:50.923673+0000","last_became_active":"2024-04-27T07:50:48.812469+0000","last_became_peered":"2024-04-27T07:50:48.812469+0000","last_unstale":"2024-04-27T07:50:50.923673+0000","last_undegraded":"2024-04-27T07:50:50.923673+0000","last_fullsized":"2024-04-27T07:50:50.923673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281980+0000","last_change":"2024-04-27T07:50:47.774587+0000","last_active":"2024-04-27T07:50:51.281980+0000","last_peered":"2024-04-27T07:50:51.281980+0000","last_clean":"2024-04-27T07:50:51.281980+0000","last_became_active":"2024-04-27T07:50:47.769083+0000","last_became_peered":"2024-04-27T07:50:47.769083+0000","last_unstale":"2024-04-27T07:50:51.281980+0000","last_undegraded":"2024-04-27T07:50:51.281980+0000","last_fullsized":"2024-04-27T07:50:51.281980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800330+0000","last_change":"2024-04-27T07:50:48.798248+0000","last_active":"2024-04-27T07:50:49.800330+0000","last_peered":"2024-04-27T07:50:49.800330+0000","last_clean":"2024-04-27T07:50:49.800330+0000","last_became_active":"2024-04-27T07:50:48.797987+0000","last_became_peered":"2024-04-27T07:50:48.797987+0000","last_unstale":"2024-04-27T07:50:49.800330+0000","last_undegraded":"2024-04-27T07:50:49.800330+0000","last_fullsized":"2024-04-27T07:50:49.800330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:28.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445367+0000","last_change":"2024-04-27T07:50:47.783980+0000","last_active":"2024-04-27T07:50:50.445367+0000","last_peered":"2024-04-27T07:50:50.445367+0000","last_clean":"2024-04-27T07:50:50.445367+0000","last_became_active":"2024-04-27T07:50:47.783127+0000","last_became_peered":"2024-04-27T07:50:47.783127+0000","last_unstale":"2024-04-27T07:50:50.445367+0000","last_undegraded":"2024-04-27T07:50:50.445367+0000","last_fullsized":"2024-04-27T07:50:50.445367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283118+0000","last_change":"2024-04-27T07:50:48.822749+0000","last_active":"2024-04-27T07:50:51.283118+0000","last_peered":"2024-04-27T07:50:51.283118+0000","last_clean":"2024-04-27T07:50:51.283118+0000","last_became_active":"2024-04-27T07:50:48.822309+0000","last_became_peered":"2024-04-27T07:50:48.822309+0000","last_unstale":"2024-04-27T07:50:51.283118+0000","last_undegraded":"2024-04-27T07:50:51.283118+0000","last_fullsized":"2024-04-27T07:50:51.283118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:43.311205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923198+0000","last_change":"2024-04-27T07:50:47.783527+0000","last_active":"2024-04-27T07:50:50.923198+0000","last_peered":"2024-04-27T07:50:50.923198+0000","last_clean":"2024-04-27T07:50:50.923198+0000","last_became_active":"2024-04-27T07:50:47.782770+0000","last_became_peered":"2024-04-27T07:50:47.782770+0000","last_unstale":"2024-04-27T07:50:50.923198+0000","last_undegraded":"2024-04-27T07:50:50.923198+0000","last_fullsized":"2024-04-27T07:50:50.923198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:30:37.164246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923078+0000","last_change":"2024-04-27T07:50:48.797984+0000","last_active":"2024-04-27T07:50:50.923078+0000","last_peered":"2024-04-27T07:50:50.923078+0000","last_clean":"2024-04-27T07:50:50.923078+0000","last_became_active":"2024-04-27T07:50:48.797194+0000","last_became_peered":"2024-04-27T07:50:48.797194+0000","last_unstale":"2024-04-27T07:50:50.923078+0000","last_undegraded":"2024-04-27T07:50:50.923078+0000","last_fullsized":"2024-04-27T07:50:50.923078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:21.654207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800940+0000","last_change":"2024-04-27T07:50:47.790007+0000","last_active":"2024-04-27T07:50:49.800940+0000","last_peered":"2024-04-27T07:50:49.800940+0000","last_clean":"2024-04-27T07:50:49.800940+0000","last_became_active":"2024-04-27T07:50:47.789403+0000","last_became_peered":"2024-04-27T07:50:47.789403+0000","last_unstale":"2024-04-27T07:50:49.800940+0000","last_undegraded":"2024-04-27T07:50:49.800940+0000","last_fullsized":"2024-04-27T07:50:49.800940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283254+0000","last_change":"2024-04-27T07:50:48.810819+0000","last_active":"2024-04-27T07:50:51.283254+0000","last_peered":"2024-04-27T07:50:51.283254+0000","last_clean":"2024-04-27T07:50:51.283254+0000","last_became_active":"2024-04-27T07:50:48.810567+0000","last_became_peered":"2024-04-27T07:50:48.810567+0000","last_unstale":"2024-04-27T07:50:51.283254+0000","last_undegraded":"2024-04-27T07:50:51.283254+0000","last_fullsized":"2024-04-27T07:50:51.283254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:50.867045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.444990+0000","last_change":"2024-04-27T07:50:47.777426+0000","last_active":"2024-04-27T07:50:50.444990+0000","last_peered":"2024-04-27T07:50:50.444990+0000","last_clean":"2024-04-27T07:50:50.444990+0000","last_became_active":"2024-04-27T07:50:47.776724+0000","last_became_peered":"2024-04-27T07:50:47.776724+0000","last_unstale":"2024-04-27T07:50:50.444990+0000","last_undegraded":"2024-04-27T07:50:50.444990+0000","last_fullsized":"2024-04-27T07:50:50.444990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:37:47.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445715+0000","last_change":"2024-04-27T07:50:48.793186+0000","last_active":"2024-04-27T07:50:50.445715+0000","last_peered":"2024-04-27T07:50:50.445715+0000","last_clean":"2024-04-27T07:50:50.445715+0000","last_became_active":"2024-04-27T07:50:48.792919+0000","last_became_peered":"2024-04-27T07:50:48.792919+0000","last_unstale":"2024-04-27T07:50:50.445715+0000","last_undegraded":"2024-04-27T07:50:50.445715+0000","last_fullsized":"2024-04-27T07:50:50.445715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:34:56.089422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797034+0000","last_change":"2024-04-27T07:50:47.770430+0000","last_active":"2024-04-27T07:50:49.797034+0000","last_peered":"2024-04-27T07:50:49.797034+0000","last_clean":"2024-04-27T07:50:49.797034+0000","last_became_active":"2024-04-27T07:50:47.770009+0000","last_became_peered":"2024-04-27T07:50:47.770009+0000","last_unstale":"2024-04-27T07:50:49.797034+0000","last_undegraded":"2024-04-27T07:50:49.797034+0000","last_fullsized":"2024-04-27T07:50:49.797034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282964+0000","last_change":"2024-04-27T07:50:48.817834+0000","last_active":"2024-04-27T07:50:51.282964+0000","last_peered":"2024-04-27T07:50:51.282964+0000","last_clean":"2024-04-27T07:50:51.282964+0000","last_became_active":"2024-04-27T07:50:48.817436+0000","last_became_peered":"2024-04-27T07:50:48.817436+0000","last_unstale":"2024-04-27T07:50:51.282964+0000","last_undegraded":"2024-04-27T07:50:51.282964+0000","last_fullsized":"2024-04-27T07:50:51.282964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:51.766684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801830+0000","last_change":"2024-04-27T07:50:47.776874+0000","last_active":"2024-04-27T07:50:49.801830+0000","last_peered":"2024-04-27T07:50:49.801830+0000","last_clean":"2024-04-27T07:50:49.801830+0000","last_became_active":"2024-04-27T07:50:47.776056+0000","last_became_peered":"2024-04-27T07:50:47.776056+0000","last_unstale":"2024-04-27T07:50:49.801830+0000","last_undegraded":"2024-04-27T07:50:49.801830+0000","last_fullsized":"2024-04-27T07:50:49.801830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:13:34.610739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283821+0000","last_change":"2024-04-27T07:50:48.812568+0000","last_active":"2024-04-27T07:50:51.283821+0000","last_peered":"2024-04-27T07:50:51.283821+0000","last_clean":"2024-04-27T07:50:51.283821+0000","last_became_active":"2024-04-27T07:50:48.811223+0000","last_became_peered":"2024-04-27T07:50:48.811223+0000","last_unstale":"2024-04-27T07:50:51.283821+0000","last_undegraded":"2024-04-27T07:50:51.283821+0000","last_fullsized":"2024-04-27T07:50:51.283821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:21:47.632417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797924+0000","last_change":"2024-04-27T07:50:47.757707+0000","last_active":"2024-04-27T07:50:49.797924+0000","last_peered":"2024-04-27T07:50:49.797924+0000","last_clean":"2024-04-27T07:50:49.797924+0000","last_became_active":"2024-04-27T07:50:47.757001+0000","last_became_peered":"2024-04-27T07:50:47.757001+0000","last_unstale":"2024-04-27T07:50:49.797924+0000","last_undegraded":"2024-04-27T07:50:49.797924+0000","last_fullsized":"2024-04-27T07:50:49.797924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:26:02.368455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283932+0000","last_change":"2024-04-27T07:50:48.820601+0000","last_active":"2024-04-27T07:50:51.283932+0000","last_peered":"2024-04-27T07:50:51.283932+0000","last_clean":"2024-04-27T07:50:51.283932+0000","last_became_active":"2024-04-27T07:50:48.817456+0000","last_became_peered":"2024-04-27T07:50:48.817456+0000","last_unstale":"2024-04-27T07:50:51.283932+0000","last_undegraded":"2024-04-27T07:50:51.283932+0000","last_fullsized":"2024-04-27T07:50:51.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:14:53.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282269+0000","last_change":"2024-04-27T07:50:47.779473+0000","last_active":"2024-04-27T07:50:51.282269+0000","last_peered":"2024-04-27T07:50:51.282269+0000","last_clean":"2024-04-27T07:50:51.282269+0000","last_became_active":"2024-04-27T07:50:47.779210+0000","last_became_peered":"2024-04-27T07:50:47.779210+0000","last_unstale":"2024-04-27T07:50:51.282269+0000","last_undegraded":"2024-04-27T07:50:51.282269+0000","last_fullsized":"2024-04-27T07:50:51.282269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:27.605869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801103+0000","last_change":"2024-04-27T07:50:48.822640+0000","last_active":"2024-04-27T07:50:49.801103+0000","last_peered":"2024-04-27T07:50:49.801103+0000","last_clean":"2024-04-27T07:50:49.801103+0000","last_became_active":"2024-04-27T07:50:48.822329+0000","last_became_peered":"2024-04-27T07:50:48.822329+0000","last_unstale":"2024-04-27T07:50:49.801103+0000","last_undegraded":"2024-04-27T07:50:49.801103+0000","last_fullsized":"2024-04-27T07:50:49.801103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:29.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797405+0000","last_change":"2024-04-27T07:50:47.767538+0000","last_active":"2024-04-27T07:50:49.797405+0000","last_peered":"2024-04-27T07:50:49.797405+0000","last_clean":"2024-04-27T07:50:49.797405+0000","last_became_active":"2024-04-27T07:50:47.767085+0000","last_became_peered":"2024-04-27T07:50:47.767085+0000","last_unstale":"2024-04-27T07:50:49.797405+0000","last_undegraded":"2024-04-27T07:50:49.797405+0000","last_fullsized":"2024-04-27T07:50:49.797405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283419+0000","last_change":"2024-04-27T07:50:48.818495+0000","last_active":"2024-04-27T07:50:51.283419+0000","last_peered":"2024-04-27T07:50:51.283419+0000","last_clean":"2024-04-27T07:50:51.283419+0000","last_became_active":"2024-04-27T07:50:48.817986+0000","last_became_peered":"2024-04-27T07:50:48.817986+0000","last_unstale":"2024-04-27T07:50:51.283419+0000","last_undegraded":"2024-04-27T07:50:51.283419+0000","last_fullsized":"2024-04-27T07:50:51.283419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:39:30.392854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282765+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:50:51.282765+0000","last_peered":"2024-04-27T07:50:51.282765+0000","last_clean":"2024-04-27T07:50:51.282765+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:50:51.282765+0000","last_undegraded":"2024-04-27T07:50:51.282765+0000","last_fullsized":"2024-04-27T07:50:51.282765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.798836+0000","last_change":"2024-04-27T07:50:47.765622+0000","last_active":"2024-04-27T07:50:49.798836+0000","last_peered":"2024-04-27T07:50:49.798836+0000","last_clean":"2024-04-27T07:50:49.798836+0000","last_became_active":"2024-04-27T07:50:47.764321+0000","last_became_peered":"2024-04-27T07:50:47.764321+0000","last_unstale":"2024-04-27T07:50:49.798836+0000","last_undegraded":"2024-04-27T07:50:49.798836+0000","last_fullsized":"2024-04-27T07:50:49.798836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:20.174472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282843+0000","last_change":"2024-04-27T07:50:48.829189+0000","last_active":"2024-04-27T07:50:51.282843+0000","last_peered":"2024-04-27T07:50:51.282843+0000","last_clean":"2024-04-27T07:50:51.282843+0000","last_became_active":"2024-04-27T07:50:48.828497+0000","last_became_peered":"2024-04-27T07:50:48.828497+0000","last_unstale":"2024-04-27T07:50:51.282843+0000","last_undegraded":"2024-04-27T07:50:51.282843+0000","last_fullsized":"2024-04-27T07:50:51.282843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282400+0000","last_change":"2024-04-27T07:50:47.801435+0000","last_active":"2024-04-27T07:50:51.282400+0000","last_peered":"2024-04-27T07:50:51.282400+0000","last_clean":"2024-04-27T07:50:51.282400+0000","last_became_active":"2024-04-27T07:50:47.800976+0000","last_became_peered":"2024-04-27T07:50:47.800976+0000","last_unstale":"2024-04-27T07:50:51.282400+0000","last_undegraded":"2024-04-27T07:50:51.282400+0000","last_fullsized":"2024-04-27T07:50:51.282400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.922633+0000","last_change":"2024-04-27T07:50:48.813418+0000","last_active":"2024-04-27T07:50:50.922633+0000","last_peered":"2024-04-27T07:50:50.922633+0000","last_clean":"2024-04-27T07:50:50.922633+0000","last_became_active":"2024-04-27T07:50:48.812995+0000","last_became_peered":"2024-04-27T07:50:48.812995+0000","last_unstale":"2024-04-27T07:50:50.922633+0000","last_undegraded":"2024-04-27T07:50:50.922633+0000","last_fullsized":"2024-04-27T07:50:50.922633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:09:23.204607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.797401+0000","last_change":"2024-04-27T07:50:47.753103+0000","last_active":"2024-04-27T07:50:49.797401+0000","last_peered":"2024-04-27T07:50:49.797401+0000","last_clean":"2024-04-27T07:50:49.797401+0000","last_became_active":"2024-04-27T07:50:47.752481+0000","last_became_peered":"2024-04-27T07:50:47.752481+0000","last_unstale":"2024-04-27T07:50:49.797401+0000","last_undegraded":"2024-04-27T07:50:49.797401+0000","last_fullsized":"2024-04-27T07:50:49.797401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281299+0000","last_change":"2024-04-27T07:50:48.806850+0000","last_active":"2024-04-27T07:50:51.281299+0000","last_peered":"2024-04-27T07:50:51.281299+0000","last_clean":"2024-04-27T07:50:51.281299+0000","last_became_active":"2024-04-27T07:50:48.801965+0000","last_became_peered":"2024-04-27T07:50:48.801965+0000","last_unstale":"2024-04-27T07:50:51.281299+0000","last_undegraded":"2024-04-27T07:50:51.281299+0000","last_fullsized":"2024-04-27T07:50:51.281299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:51:42.279913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801000+0000","last_change":"2024-04-27T07:50:47.779751+0000","last_active":"2024-04-27T07:50:49.801000+0000","last_peered":"2024-04-27T07:50:49.801000+0000","last_clean":"2024-04-27T07:50:49.801000+0000","last_became_active":"2024-04-27T07:50:47.778788+0000","last_became_peered":"2024-04-27T07:50:47.778788+0000","last_unstale":"2024-04-27T07:50:49.801000+0000","last_undegraded":"2024-04-27T07:50:49.801000+0000","last_fullsized":"2024-04-27T07:50:49.801000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801821+0000","last_change":"2024-04-27T07:50:48.821201+0000","last_active":"2024-04-27T07:50:49.801821+0000","last_peered":"2024-04-27T07:50:49.801821+0000","last_clean":"2024-04-27T07:50:49.801821+0000","last_became_active":"2024-04-27T07:50:48.820923+0000","last_became_peered":"2024-04-27T07:50:48.820923+0000","last_unstale":"2024-04-27T07:50:49.801821+0000","last_undegraded":"2024-04-27T07:50:49.801821+0000","last_fullsized":"2024-04-27T07:50:49.801821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:25.284413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281927+0000","last_change":"2024-04-27T07:50:47.769907+0000","last_active":"2024-04-27T07:50:51.281927+0000","last_peered":"2024-04-27T07:50:51.281927+0000","last_clean":"2024-04-27T07:50:51.281927+0000","last_became_active":"2024-04-27T07:50:47.769126+0000","last_became_peered":"2024-04-27T07:50:47.769126+0000","last_unstale":"2024-04-27T07:50:51.281927+0000","last_undegraded":"2024-04-27T07:50:51.281927+0000","last_fullsized":"2024-04-27T07:50:51.281927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:01.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.800281+0000","last_change":"2024-04-27T07:50:48.804829+0000","last_active":"2024-04-27T07:50:49.800281+0000","last_peered":"2024-04-27T07:50:49.800281+0000","last_clean":"2024-04-27T07:50:49.800281+0000","last_became_active":"2024-04-27T07:50:48.804092+0000","last_became_peered":"2024-04-27T07:50:48.804092+0000","last_unstale":"2024-04-27T07:50:49.800281+0000","last_undegraded":"2024-04-27T07:50:49.800281+0000","last_fullsized":"2024-04-27T07:50:49.800281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283754+0000","last_change":"2024-04-27T07:50:47.801329+0000","last_active":"2024-04-27T07:50:51.283754+0000","last_peered":"2024-04-27T07:50:51.283754+0000","last_clean":"2024-04-27T07:50:51.283754+0000","last_became_active":"2024-04-27T07:50:47.800993+0000","last_became_peered":"2024-04-27T07:50:47.800993+0000","last_unstale":"2024-04-27T07:50:51.283754+0000","last_undegraded":"2024-04-27T07:50:51.283754+0000","last_fullsized":"2024-04-27T07:50:51.283754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:52:57.849530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283493+0000","last_change":"2024-04-27T07:50:48.795907+0000","last_active":"2024-04-27T07:50:51.283493+0000","last_peered":"2024-04-27T07:50:51.283493+0000","last_clean":"2024-04-27T07:50:51.283493+0000","last_became_active":"2024-04-27T07:50:48.795249+0000","last_became_peered":"2024-04-27T07:50:48.795249+0000","last_unstale":"2024-04-27T07:50:51.283493+0000","last_undegraded":"2024-04-27T07:50:51.283493+0000","last_fullsized":"2024-04-27T07:50:51.283493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282216+0000","last_change":"2024-04-27T07:50:47.774585+0000","last_active":"2024-04-27T07:50:51.282216+0000","last_peered":"2024-04-27T07:50:51.282216+0000","last_clean":"2024-04-27T07:50:51.282216+0000","last_became_active":"2024-04-27T07:50:47.774213+0000","last_became_peered":"2024-04-27T07:50:47.774213+0000","last_unstale":"2024-04-27T07:50:51.282216+0000","last_undegraded":"2024-04-27T07:50:51.282216+0000","last_fullsized":"2024-04-27T07:50:51.282216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:18.170939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283467+0000","last_change":"2024-04-27T07:50:48.797717+0000","last_active":"2024-04-27T07:50:51.283467+0000","last_peered":"2024-04-27T07:50:51.283467+0000","last_clean":"2024-04-27T07:50:51.283467+0000","last_became_active":"2024-04-27T07:50:48.797011+0000","last_became_peered":"2024-04-27T07:50:48.797011+0000","last_unstale":"2024-04-27T07:50:51.283467+0000","last_undegraded":"2024-04-27T07:50:51.283467+0000","last_fullsized":"2024-04-27T07:50:51.283467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:18.028993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445562+0000","last_change":"2024-04-27T07:50:47.796196+0000","last_active":"2024-04-27T07:50:50.445562+0000","last_peered":"2024-04-27T07:50:50.445562+0000","last_clean":"2024-04-27T07:50:50.445562+0000","last_became_active":"2024-04-27T07:50:47.795881+0000","last_became_peered":"2024-04-27T07:50:47.795881+0000","last_unstale":"2024-04-27T07:50:50.445562+0000","last_undegraded":"2024-04-27T07:50:50.445562+0000","last_fullsized":"2024-04-27T07:50:50.445562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:30:11.967127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923685+0000","last_change":"2024-04-27T07:50:48.804448+0000","last_active":"2024-04-27T07:50:50.923685+0000","last_peered":"2024-04-27T07:50:50.923685+0000","last_clean":"2024-04-27T07:50:50.923685+0000","last_became_active":"2024-04-27T07:50:48.803586+0000","last_became_peered":"2024-04-27T07:50:48.803586+0000","last_unstale":"2024-04-27T07:50:50.923685+0000","last_undegraded":"2024-04-27T07:50:50.923685+0000","last_fullsized":"2024-04-27T07:50:50.923685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:28.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282373+0000","last_change":"2024-04-27T07:50:47.796755+0000","last_active":"2024-04-27T07:50:51.282373+0000","last_peered":"2024-04-27T07:50:51.282373+0000","last_clean":"2024-04-27T07:50:51.282373+0000","last_became_active":"2024-04-27T07:50:47.796096+0000","last_became_peered":"2024-04-27T07:50:47.796096+0000","last_unstale":"2024-04-27T07:50:51.282373+0000","last_undegraded":"2024-04-27T07:50:51.282373+0000","last_fullsized":"2024-04-27T07:50:51.282373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:59:42.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799129+0000","last_change":"2024-04-27T07:50:48.792778+0000","last_active":"2024-04-27T07:50:49.799129+0000","last_peered":"2024-04-27T07:50:49.799129+0000","last_clean":"2024-04-27T07:50:49.799129+0000","last_became_active":"2024-04-27T07:50:48.791993+0000","last_became_peered":"2024-04-27T07:50:48.791993+0000","last_unstale":"2024-04-27T07:50:49.799129+0000","last_undegraded":"2024-04-27T07:50:49.799129+0000","last_fullsized":"2024-04-27T07:50:49.799129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:02.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281677+0000","last_change":"2024-04-27T07:50:47.785296+0000","last_active":"2024-04-27T07:50:51.281677+0000","last_peered":"2024-04-27T07:50:51.281677+0000","last_clean":"2024-04-27T07:50:51.281677+0000","last_became_active":"2024-04-27T07:50:47.784874+0000","last_became_peered":"2024-04-27T07:50:47.784874+0000","last_unstale":"2024-04-27T07:50:51.281677+0000","last_undegraded":"2024-04-27T07:50:51.281677+0000","last_fullsized":"2024-04-27T07:50:51.281677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:24:19.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281489+0000","last_change":"2024-04-27T07:50:48.809822+0000","last_active":"2024-04-27T07:50:51.281489+0000","last_peered":"2024-04-27T07:50:51.281489+0000","last_clean":"2024-04-27T07:50:51.281489+0000","last_became_active":"2024-04-27T07:50:48.809495+0000","last_became_peered":"2024-04-27T07:50:48.809495+0000","last_unstale":"2024-04-27T07:50:51.281489+0000","last_undegraded":"2024-04-27T07:50:51.281489+0000","last_fullsized":"2024-04-27T07:50:51.281489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:53.932764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.281879+0000","last_change":"2024-04-27T07:50:47.777251+0000","last_active":"2024-04-27T07:50:51.281879+0000","last_peered":"2024-04-27T07:50:51.281879+0000","last_clean":"2024-04-27T07:50:51.281879+0000","last_became_active":"2024-04-27T07:50:47.776893+0000","last_became_peered":"2024-04-27T07:50:47.776893+0000","last_unstale":"2024-04-27T07:50:51.281879+0000","last_undegraded":"2024-04-27T07:50:51.281879+0000","last_fullsized":"2024-04-27T07:50:51.281879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:35:24.275883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282509+0000","last_change":"2024-04-27T07:50:48.821019+0000","last_active":"2024-04-27T07:50:51.282509+0000","last_peered":"2024-04-27T07:50:51.282509+0000","last_clean":"2024-04-27T07:50:51.282509+0000","last_became_active":"2024-04-27T07:50:48.820606+0000","last_became_peered":"2024-04-27T07:50:48.820606+0000","last_unstale":"2024-04-27T07:50:51.282509+0000","last_undegraded":"2024-04-27T07:50:51.282509+0000","last_fullsized":"2024-04-27T07:50:51.282509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:25.349473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283602+0000","last_change":"2024-04-27T07:50:47.800388+0000","last_active":"2024-04-27T07:50:51.283602+0000","last_peered":"2024-04-27T07:50:51.283602+0000","last_clean":"2024-04-27T07:50:51.283602+0000","last_became_active":"2024-04-27T07:50:47.799604+0000","last_became_peered":"2024-04-27T07:50:47.799604+0000","last_unstale":"2024-04-27T07:50:51.283602+0000","last_undegraded":"2024-04-27T07:50:51.283602+0000","last_fullsized":"2024-04-27T07:50:51.283602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:34:31.234332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.445740+0000","last_change":"2024-04-27T07:50:48.810527+0000","last_active":"2024-04-27T07:50:50.445740+0000","last_peered":"2024-04-27T07:50:50.445740+0000","last_clean":"2024-04-27T07:50:50.445740+0000","last_became_active":"2024-04-27T07:50:48.810203+0000","last_became_peered":"2024-04-27T07:50:48.810203+0000","last_unstale":"2024-04-27T07:50:50.445740+0000","last_undegraded":"2024-04-27T07:50:50.445740+0000","last_fullsized":"2024-04-27T07:50:50.445740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:59.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.799691+0000","last_change":"2024-04-27T07:50:48.787926+0000","last_active":"2024-04-27T07:50:49.799691+0000","last_peered":"2024-04-27T07:50:49.799691+0000","last_clean":"2024-04-27T07:50:49.799691+0000","last_became_active":"2024-04-27T07:50:48.787213+0000","last_became_peered":"2024-04-27T07:50:48.787213+0000","last_unstale":"2024-04-27T07:50:49.799691+0000","last_undegraded":"2024-04-27T07:50:49.799691+0000","last_fullsized":"2024-04-27T07:50:49.799691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:25:28.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282943+0000","last_change":"2024-04-27T07:50:47.799584+0000","last_active":"2024-04-27T07:50:51.282943+0000","last_peered":"2024-04-27T07:50:51.282943+0000","last_clean":"2024-04-27T07:50:51.282943+0000","last_became_active":"2024-04-27T07:50:47.799182+0000","last_became_peered":"2024-04-27T07:50:47.799182+0000","last_unstale":"2024-04-27T07:50:51.282943+0000","last_undegraded":"2024-04-27T07:50:51.282943+0000","last_fullsized":"2024-04-27T07:50:51.282943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:31:29.207109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:49.801755+0000","last_change":"2024-04-27T07:50:48.798910+0000","last_active":"2024-04-27T07:50:49.801755+0000","last_peered":"2024-04-27T07:50:49.801755+0000","last_clean":"2024-04-27T07:50:49.801755+0000","last_became_active":"2024-04-27T07:50:48.798376+0000","last_became_peered":"2024-04-27T07:50:48.798376+0000","last_unstale":"2024-04-27T07:50:49.801755+0000","last_undegraded":"2024-04-27T07:50:49.801755+0000","last_fullsized":"2024-04-27T07:50:49.801755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:17:52.504141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.283271+0000","last_change":"2024-04-27T07:50:47.798503+0000","last_active":"2024-04-27T07:50:51.283271+0000","last_peered":"2024-04-27T07:50:51.283271+0000","last_clean":"2024-04-27T07:50:51.283271+0000","last_became_active":"2024-04-27T07:50:47.798171+0000","last_became_peered":"2024-04-27T07:50:47.798171+0000","last_unstale":"2024-04-27T07:50:51.283271+0000","last_undegraded":"2024-04-27T07:50:51.283271+0000","last_fullsized":"2024-04-27T07:50:51.283271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:16.910153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923642+0000","last_change":"2024-04-27T07:50:48.818082+0000","last_active":"2024-04-27T07:50:50.923642+0000","last_peered":"2024-04-27T07:50:50.923642+0000","last_clean":"2024-04-27T07:50:50.923642+0000","last_became_active":"2024-04-27T07:50:48.813287+0000","last_became_peered":"2024-04-27T07:50:48.813287+0000","last_unstale":"2024-04-27T07:50:50.923642+0000","last_undegraded":"2024-04-27T07:50:50.923642+0000","last_fullsized":"2024-04-27T07:50:50.923642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:27.571419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282997+0000","last_change":"2024-04-27T07:50:47.770647+0000","last_active":"2024-04-27T07:50:51.282997+0000","last_peered":"2024-04-27T07:50:51.282997+0000","last_clean":"2024-04-27T07:50:51.282997+0000","last_became_active":"2024-04-27T07:50:47.770223+0000","last_became_peered":"2024-04-27T07:50:47.770223+0000","last_unstale":"2024-04-27T07:50:51.282997+0000","last_undegraded":"2024-04-27T07:50:51.282997+0000","last_fullsized":"2024-04-27T07:50:51.282997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:00:05.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:51.282890+0000","last_change":"2024-04-27T07:50:48.828744+0000","last_active":"2024-04-27T07:50:51.282890+0000","last_peered":"2024-04-27T07:50:51.282890+0000","last_clean":"2024-04-27T07:50:51.282890+0000","last_became_active":"2024-04-27T07:50:48.810916+0000","last_became_peered":"2024-04-27T07:50:48.810916+0000","last_unstale":"2024-04-27T07:50:51.282890+0000","last_undegraded":"2024-04-27T07:50:51.282890+0000","last_fullsized":"2024-04-27T07:50:51.282890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:47.736491+0000","last_clean_scrub_stamp":"2024-04-27T07:50:47.736491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:00.657224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-27T07:50:50.923774+0000","last_change":"2024-04-27T07:50:47.785683+0000","last_active":"2024-04-27T07:50:50.923774+0000","last_peered":"2024-04-27T07:50:50.923774+0000","last_clean":"2024-04-27T07:50:50.923774+0000","last_became_active":"2024-04-27T07:50:47.784993+0000","last_became_peered":"2024-04-27T07:50:47.784993+0000","last_unstale":"2024-04-27T07:50:50.923774+0000","last_undegraded":"2024-04-27T07:50:50.923774+0000","last_fullsized":"2024-04-27T07:50:50.923774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:46.727961+0000","last_clean_scrub_stamp":"2024-04-27T07:50:46.727961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:49.046466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:51:02.254 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T07:51:02.254 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T07:51:02.254 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-27T07:51:02.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-27T07:51:02.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.411+0000 7f2e391b1640 1 Processor -- start 2024-04-27T07:51:02.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.411+0000 7f2e391b1640 1 -- start start 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e391b1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2e340b4610 0x7f2e3406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e391b1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340b6750 con 0x7f2e3406c110 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e391b1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340b6190 con 0x7f2e34148230 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e391b1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3406c580 con 0x7f2e340b4610 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2e340b4610 0x7f2e3406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2e340b4610 0x7f2e3406bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40842/0 (socket says 172.21.15.22:40842) 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3359e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2e3406c110 0x7f2e340cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41236/0 (socket says 172.21.15.22:41236) 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 -- 172.21.15.22:0/3986381715 learned_addr learned my addr 172.21.15.22:0/3986381715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299408120 0 0) 0x7f2e340b6750 con 0x7f2e3406c110 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e1c003530 con 0x7f2e3406c110 2024-04-27T07:51:02.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 -- 172.21.15.22:0/3986381715 >> v1:172.21.15.183:6789/0 conn(0x7f2e34148230 legacy=0x7f2e340b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1942004965 0 0) 0x7f2e1c003530 con 0x7f2e3406c110 2024-04-27T07:51:02.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 -- 172.21.15.22:0/3986381715 >> v1:172.21.15.22:6789/0 conn(0x7f2e3406c110 legacy=0x7f2e340cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 -- 172.21.15.22:0/3986381715 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340b6470 con 0x7f2e340b4610 2024-04-27T07:51:02.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e3259c640 1 --2- 172.21.15.22:0/3986381715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e3406bbd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009870 tx=0x7f2e2801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514a80879f4fd7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280b0070 con 0x7f2e340b4610 2024-04-27T07:51:02.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e280ab400 con 0x7f2e340b4610 2024-04-27T07:51:02.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.415+0000 7f2e31d9b640 1 -- 172.21.15.22:0/3986381715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c001680 con 0x7f2e340b4610 2024-04-27T07:51:02.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3986381715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 msgr2=0x7f2e3406bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3986381715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e3406bbd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009870 tx=0x7f2e2801c480 comp rx=0 tx=0).stop 2024-04-27T07:51:02.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3986381715 shutdown_connections 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3986381715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e3406bbd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3986381715 >> 172.21.15.22:0/3986381715 conn(0x7f2e3409aa30 msgr2=0x7f2e340b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3986381715 shutdown_connections 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3986381715 wait complete. 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 Processor -- start 2024-04-27T07:51:02.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- start start 2024-04-27T07:51:02.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 0x7f2e3415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e34148230 0x7f2e340b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d02b0 con 0x7f2e340b4610 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d0130 con 0x7f2e3406c110 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3416c590 con 0x7f2e34148230 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40846/0 (socket says 172.21.15.22:40846) 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 -- 172.21.15.22:0/3958468594 learned_addr learned my addr 172.21.15.22:0/3958468594 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e32d9d640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 0x7f2e3415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3359e640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e34148230 0x7f2e340b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 -- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e34148230 msgr2=0x7f2e340b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e34148230 0x7f2e340b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 -- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 msgr2=0x7f2e3415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 0x7f2e3415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 -- 172.21.15.22:0/3958468594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3415fb10 con 0x7f2e340b4610 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e32d9d640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 0x7f2e3415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:02.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e3259c640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e280ae040 tx=0x7f2e28019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280bb070 con 0x7f2e340b4610 2024-04-27T07:51:02.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28004230 con 0x7f2e340b4610 2024-04-27T07:51:02.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280b0040 con 0x7f2e340b4610 2024-04-27T07:51:02.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3415fdf0 con 0x7f2e340b4610 2024-04-27T07:51:02.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.419+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341603b0 con 0x7f2e340b4610 2024-04-27T07:51:02.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e34069340 con 0x7f2e340b4610 2024-04-27T07:51:02.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e28003620 con 0x7f2e340b4610 2024-04-27T07:51:02.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e1b7fe640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 0x7f2e2006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2e28120220 con 0x7f2e340b4610 2024-04-27T07:51:02.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e32d9d640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 0x7f2e2006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.423+0000 7f2e32d9d640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 0x7f2e2006a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e24007fd0 tx=0x7f2e24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.427+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e28016070 con 0x7f2e340b4610 2024-04-27T07:51:02.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.615+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2e34140e00 con 0x7f2e340b4610 2024-04-27T07:51:02.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.615+0000 7f2e1b7fe640 1 -- 172.21.15.22:0/3958468594 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2e280f07e0 con 0x7f2e340b4610 2024-04-27T07:51:02.622 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:02.622 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 msgr2=0x7f2e2006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 0x7f2e2006a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e24007fd0 tx=0x7f2e24009040 comp rx=0 tx=0).stop 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 msgr2=0x7f2e341616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e280ae040 tx=0x7f2e28019220 comp rx=0 tx=0).stop 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 shutdown_connections 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2e200683b0 0x7f2e2006a870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e34148230 0x7f2e340b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e340b4610 0x7f2e341616c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 --2- 172.21.15.22:0/3958468594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e3406c110 0x7f2e3415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 >> 172.21.15.22:0/3958468594 conn(0x7f2e3409aa30 msgr2=0x7f2e34070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 shutdown_connections 2024-04-27T07:51:02.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.619+0000 7f2e391b1640 1 -- 172.21.15.22:0/3958468594 wait complete. 2024-04-27T07:51:02.640 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-27T07:51:02.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 Processor -- start 2024-04-27T07:51:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 -- start start 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f38f4072740 0x7f38f4072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f406e050 con 0x7f38f4145a00 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f40ceae0 con 0x7f38f41454b0 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f9bac640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40cedc0 con 0x7f38f4072740 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f38f4072740 0x7f38f4072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f38f4145a00 0x7f38f415cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41242/0 (socket says 172.21.15.22:41242) 2024-04-27T07:51:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f3fff640 1 -- 172.21.15.22:0/725033287 learned_addr learned my addr 172.21.15.22:0/725033287 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:02.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f2ffd640 1 -- 172.21.15.22:0/725033287 >> v1:172.21.15.183:6789/0 conn(0x7f38f41454b0 legacy=0x7f38f4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:02.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f2ffd640 1 -- 172.21.15.22:0/725033287 >> v1:172.21.15.22:6789/0 conn(0x7f38f4145a00 legacy=0x7f38f415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f2ffd640 1 -- 172.21.15.22:0/725033287 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f406e310 con 0x7f38f4072740 2024-04-27T07:51:02.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.755+0000 7f38f27fc640 1 -- 172.21.15.22:0/725033287 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923433006 0 0) 0x7f38f406e050 con 0x7f38f4145a00 2024-04-27T07:51:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 --2- 172.21.15.22:0/725033287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f4072740 0x7f38f4072fa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097c0 tx=0x7f38e401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11e11e135daa49e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f27fc640 1 -- 172.21.15.22:0/725033287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40b0070 con 0x7f38f4072740 2024-04-27T07:51:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f27fc640 1 -- 172.21.15.22:0/725033287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e40ab600 con 0x7f38f4072740 2024-04-27T07:51:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f27fc640 1 -- 172.21.15.22:0/725033287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40ab9d0 con 0x7f38f4072740 2024-04-27T07:51:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- 172.21.15.22:0/725033287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f4072740 msgr2=0x7f38f4072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 --2- 172.21.15.22:0/725033287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f4072740 0x7f38f4072fa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097c0 tx=0x7f38e401c460 comp rx=0 tx=0).stop 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- 172.21.15.22:0/725033287 shutdown_connections 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 --2- 172.21.15.22:0/725033287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f4072740 0x7f38f4072fa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- 172.21.15.22:0/725033287 >> 172.21.15.22:0/725033287 conn(0x7f38f4067f10 msgr2=0x7f38f406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- 172.21.15.22:0/725033287 shutdown_connections 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- 172.21.15.22:0/725033287 wait complete. 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 Processor -- start 2024-04-27T07:51:02.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- start start 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38f4072740 0x7f38f40c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38f4145a00 0x7f38f415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415d9a0 con 0x7f38f41454b0 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415d820 con 0x7f38f4145a00 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f9bac640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38f416ac80 con 0x7f38f4072740 2024-04-27T07:51:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40870/0 (socket says 172.21.15.22:40870) 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 -- 172.21.15.22:0/1915828285 learned_addr learned my addr 172.21.15.22:0/1915828285 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 -- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38f4072740 msgr2=0x7f38f40c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38f4072740 0x7f38f40c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f2ffd640 1 -- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38f4145a00 msgr2=0x7f38f415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.759+0000 7f38f3fff640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38f4145a00 0x7f38f415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f2ffd640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38f4145a00 0x7f38f415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f2ffd640 1 -- 172.21.15.22:0/1915828285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40ae600 con 0x7f38f41454b0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f2ffd640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097c0 tx=0x7f38e401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40bd070 con 0x7f38f41454b0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e4004230 con 0x7f38f41454b0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f40ae8e0 con 0x7f38f41454b0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40b0040 con 0x7f38f41454b0 2024-04-27T07:51:02.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f40aeea0 con 0x7f38f41454b0 2024-04-27T07:51:02.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f4068c30 con 0x7f38f41454b0 2024-04-27T07:51:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38e40c0070 con 0x7f38f41454b0 2024-04-27T07:51:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 0x7f38e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f38e41204c0 con 0x7f38f41454b0 2024-04-27T07:51:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f37fe640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 0x7f38e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.763+0000 7f38f37fe640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 0x7f38e006a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b28f0 tx=0x7f38e80079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:02.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.767+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38e4016050 con 0x7f38f41454b0 2024-04-27T07:51:02.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.923+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38f413f6d0 con 0x7f38f41454b0 2024-04-27T07:51:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.923+0000 7f38dbfff640 1 -- 172.21.15.22:0/1915828285 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f38e40f0a80 con 0x7f38f41454b0 2024-04-27T07:51:02.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:02.931 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 msgr2=0x7f38e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 0x7f38e006a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b28f0 tx=0x7f38e80079c0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 msgr2=0x7f38f40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097c0 tx=0x7f38e401aca0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 shutdown_connections 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f38e0068380 0x7f38e006a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38f4145a00 0x7f38f415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38f41454b0 0x7f38f40b4f60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 --2- 172.21.15.22:0/1915828285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38f4072740 0x7f38f40c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 >> 172.21.15.22:0/1915828285 conn(0x7f38f4067f10 msgr2=0x7f38f4140c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 shutdown_connections 2024-04-27T07:51:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:02.927+0000 7f38f9bac640 1 -- 172.21.15.22:0/1915828285 wait complete. 2024-04-27T07:51:02.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:02.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:03.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 Processor -- start 2024-04-27T07:51:03.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 -- start start 2024-04-27T07:51:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7effdc15af30 0x7effdc06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7effdc072f10 0x7effdc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effdc15c2a0 con 0x7effdc0af990 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7effdc15b4e0 con 0x7effdc072f10 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe3f6b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7effdc15b7c0 con 0x7effdc15af30 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7effdc072f10 0x7effdc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe1506640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7effdc0af990 0x7effdc072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41244/0 (socket says 172.21.15.22:41244) 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7effdc072f10 0x7effdc0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40872/0 (socket says 172.21.15.22:40872) 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe1506640 1 -- 172.21.15.22:0/1546088077 learned_addr learned my addr 172.21.15.22:0/1546088077 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe1d07640 1 --2- 172.21.15.22:0/1546088077 >> v2:172.21.15.183:3300/0 conn(0x7effdc15af30 0x7effdc06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 -- 172.21.15.22:0/1546088077 >> v2:172.21.15.183:3300/0 conn(0x7effdc15af30 msgr2=0x7effdc06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 --2- 172.21.15.22:0/1546088077 >> v2:172.21.15.183:3300/0 conn(0x7effdc15af30 0x7effdc06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 -- 172.21.15.22:0/1546088077 >> v1:172.21.15.22:6789/0 conn(0x7effdc0af990 legacy=0x7effdc072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 -- 172.21.15.22:0/1546088077 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc15baa0 con 0x7effdc072f10 2024-04-27T07:51:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.059+0000 7effe2508640 1 --2- 172.21.15.22:0/1546088077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc072f10 0x7effdc0b34d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effdc050ea0 tx=0x7effd401d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1136d13e86ebcae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe0d05640 1 -- 172.21.15.22:0/1546088077 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40a6620 con 0x7effdc072f10 2024-04-27T07:51:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe0d05640 1 -- 172.21.15.22:0/1546088077 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd40a67c0 con 0x7effdc072f10 2024-04-27T07:51:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe0d05640 1 -- 172.21.15.22:0/1546088077 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40a6b90 con 0x7effdc072f10 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/1546088077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc072f10 msgr2=0x7effdc0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- 172.21.15.22:0/1546088077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc072f10 0x7effdc0b34d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effdc050ea0 tx=0x7effd401d9c0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/1546088077 shutdown_connections 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- 172.21.15.22:0/1546088077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc072f10 0x7effdc0b34d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- 172.21.15.22:0/1546088077 >> v2:172.21.15.183:3300/0 conn(0x7effdc15af30 0x7effdc06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/1546088077 >> 172.21.15.22:0/1546088077 conn(0x7effdc0b21d0 msgr2=0x7effdc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/1546088077 shutdown_connections 2024-04-27T07:51:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/1546088077 wait complete. 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 Processor -- start 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- start start 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc072f10 0x7effdc1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effdc0af990 0x7effdc13fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7effdc167a70 con 0x7effdc15af30 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effdc0b4430 con 0x7effdc072f10 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7effdc167bf0 con 0x7effdc0af990 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40874/0 (socket says 172.21.15.22:40874) 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 -- 172.21.15.22:0/2640126092 learned_addr learned my addr 172.21.15.22:0/2640126092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe1506640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effdc0af990 0x7effdc13fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 -- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effdc0af990 msgr2=0x7effdc13fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effdc0af990 0x7effdc13fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 -- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc072f10 msgr2=0x7effdc1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe1d07640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc072f10 0x7effdc1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc072f10 0x7effdc1527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 -- 172.21.15.22:0/2640126092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc0a9a00 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe2508640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd40b0040 tx=0x7effd401a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd400a560 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd40bd070 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40b7ad0 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc0a9ce0 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.063+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc0aa2a0 con 0x7effdc15af30 2024-04-27T07:51:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effdc04ff10 con 0x7effdc15af30 2024-04-27T07:51:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7effd40b7c70 con 0x7effdc15af30 2024-04-27T07:51:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effd27fc640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 0x7effc006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7effd41219d0 con 0x7effdc15af30 2024-04-27T07:51:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effe1d07640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 0x7effc006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.067+0000 7effe1d07640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 0x7effc006a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effd80096f0 tx=0x7effd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.071+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effd4017640 con 0x7effdc15af30 2024-04-27T07:51:03.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.227+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effdc0cdd00 con 0x7effdc15af30 2024-04-27T07:51:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.227+0000 7effd27fc640 1 -- 172.21.15.22:0/2640126092 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7effd4008360 con 0x7effdc15af30 2024-04-27T07:51:03.233 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 msgr2=0x7effc006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 0x7effc006a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effd80096f0 tx=0x7effd8008040 comp rx=0 tx=0).stop 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 msgr2=0x7effdc0ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd40b0040 tx=0x7effd401a760 comp rx=0 tx=0).stop 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 shutdown_connections 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7effc0068380 0x7effc006a840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effdc15af30 0x7effdc0ac9d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effdc0af990 0x7effdc13fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 --2- 172.21.15.22:0/2640126092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc072f10 0x7effdc1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 >> 172.21.15.22:0/2640126092 conn(0x7effdc0b21d0 msgr2=0x7effdc0ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 shutdown_connections 2024-04-27T07:51:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.231+0000 7effe3f6b640 1 -- 172.21.15.22:0/2640126092 wait complete. 2024-04-27T07:51:03.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.371+0000 7f7add541640 1 Processor -- start 2024-04-27T07:51:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.371+0000 7f7add541640 1 -- start start 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.371+0000 7f7add541640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ad815af30 0x7f7ad806f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.371+0000 7f7add541640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad815c2a0 con 0x7f7ad80af990 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad815b4e0 con 0x7f7ad8072f10 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad815b7c0 con 0x7f7ad815af30 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ad815af30 0x7f7ad806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ad815af30 0x7f7ad806f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40890/0 (socket says 172.21.15.22:40890) 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/1453153910 learned_addr learned my addr 172.21.15.22:0/1453153910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/1453153910 >> v1:172.21.15.183:6789/0 conn(0x7f7ad8072f10 legacy=0x7f7ad80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/1453153910 >> v1:172.21.15.22:6789/0 conn(0x7f7ad80af990 legacy=0x7f7ad8072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/1453153910 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad815baa0 con 0x7f7ad815af30 2024-04-27T07:51:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad6ffd640 1 --2- 172.21.15.22:0/1453153910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815af30 0x7f7ad806f490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac801d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b28061554c0906 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad5ffb640 1 -- 172.21.15.22:0/1453153910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b1070 con 0x7f7ad815af30 2024-04-27T07:51:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad5ffb640 1 -- 172.21.15.22:0/1453153910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac80ac600 con 0x7f7ad815af30 2024-04-27T07:51:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7ad5ffb640 1 -- 172.21.15.22:0/1453153910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80ac9d0 con 0x7f7ad815af30 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- 172.21.15.22:0/1453153910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815af30 msgr2=0x7f7ad806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 --2- 172.21.15.22:0/1453153910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815af30 0x7f7ad806f490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac801d5a0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- 172.21.15.22:0/1453153910 shutdown_connections 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 --2- 172.21.15.22:0/1453153910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815af30 0x7f7ad806f490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- 172.21.15.22:0/1453153910 >> 172.21.15.22:0/1453153910 conn(0x7f7ad80b21d0 msgr2=0x7f7ad80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- 172.21.15.22:0/1453153910 shutdown_connections 2024-04-27T07:51:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.375+0000 7f7add541640 1 -- 172.21.15.22:0/1453153910 wait complete. 2024-04-27T07:51:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 Processor -- start 2024-04-27T07:51:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- start start 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad80af990 0x7f7ad81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815af30 0x7f7ad8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8168c00 con 0x7f7ad8072f10 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8168a80 con 0x7f7ad815af30 2024-04-27T07:51:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad8168d80 con 0x7f7ad80af990 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40900/0 (socket says 172.21.15.22:40900) 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/65374745 learned_addr learned my addr 172.21.15.22:0/65374745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad80af990 msgr2=0x7f7ad81562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad80af990 0x7f7ad81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815af30 msgr2=0x7f7ad8152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815af30 0x7f7ad8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 -- 172.21.15.22:0/65374745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad8153480 con 0x7f7ad8072f10 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7ad6ffd640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ac801a1b0 tx=0x7f7ac801a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b1040 con 0x7f7ad8072f10 2024-04-27T07:51:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac80ba070 con 0x7f7ad8072f10 2024-04-27T07:51:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad8155510 con 0x7f7ad8072f10 2024-04-27T07:51:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80abbb0 con 0x7f7ad8072f10 2024-04-27T07:51:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad8154360 con 0x7f7ad8072f10 2024-04-27T07:51:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad804ff10 con 0x7f7ad8072f10 2024-04-27T07:51:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ac80bf460 con 0x7f7ad8072f10 2024-04-27T07:51:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.379+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7ac8120950 con 0x7f7ad8072f10 2024-04-27T07:51:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.383+0000 7f7ad67fc640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.383+0000 7f7ad67fc640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000f70 tx=0x7f7acc0079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.387+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ac8017220 con 0x7f7ad8072f10 2024-04-27T07:51:03.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.539+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ad80ccc20 con 0x7f7ad8072f10 2024-04-27T07:51:03.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.539+0000 7f7abf7fe640 1 -- 172.21.15.22:0/65374745 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7ac80bc310 con 0x7f7ad8072f10 2024-04-27T07:51:03.545 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 msgr2=0x7f7ac406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000f70 tx=0x7f7acc0079e0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 msgr2=0x7f7ad80cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ac801a1b0 tx=0x7f7ac801a340 comp rx=0 tx=0).stop 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 shutdown_connections 2024-04-27T07:51:03.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815af30 0x7f7ad8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad80af990 0x7f7ad81562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 --2- 172.21.15.22:0/65374745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad8072f10 0x7f7ad80cf080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 >> 172.21.15.22:0/65374745 conn(0x7f7ad80b21d0 msgr2=0x7f7ad80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 shutdown_connections 2024-04-27T07:51:03.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.543+0000 7f7add541640 1 -- 172.21.15.22:0/65374745 wait complete. 2024-04-27T07:51:03.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 Processor -- start 2024-04-27T07:51:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 -- start start 2024-04-27T07:51:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7428071f70 0x7f7428072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742806e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7428072890 con 0x7f742806ec80 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f74280ce0d0 con 0x7f7428071f70 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742f888640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74280cd8c0 con 0x7f74280cffe0 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7428071f70 0x7f7428072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7428071f70 0x7f7428072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40906/0 (socket says 172.21.15.22:40906) 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 -- 172.21.15.22:0/3340855063 learned_addr learned my addr 172.21.15.22:0/3340855063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742ce23640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 -- 172.21.15.22:0/3340855063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 msgr2=0x7f742806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 -- 172.21.15.22:0/3340855063 >> v1:172.21.15.183:6789/0 conn(0x7f742806ec80 legacy=0x7f742806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742d624640 1 -- 172.21.15.22:0/3340855063 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74280cdb10 con 0x7f7428071f70 2024-04-27T07:51:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.679+0000 7f742ce23640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742d624640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7428071f70 0x7f7428072350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7418009870 tx=0x7f741801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdad59970911643 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f741ffff640 1 -- 172.21.15.22:0/3340855063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180b0070 con 0x7f7428071f70 2024-04-27T07:51:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f741ffff640 1 -- 172.21.15.22:0/3340855063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74180ab600 con 0x7f7428071f70 2024-04-27T07:51:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f741ffff640 1 -- 172.21.15.22:0/3340855063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180ab9d0 con 0x7f7428071f70 2024-04-27T07:51:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- 172.21.15.22:0/3340855063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7428071f70 msgr2=0x7f7428072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7428071f70 0x7f7428072350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7418009870 tx=0x7f741801c490 comp rx=0 tx=0).stop 2024-04-27T07:51:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- 172.21.15.22:0/3340855063 shutdown_connections 2024-04-27T07:51:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- 172.21.15.22:0/3340855063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7428071f70 0x7f7428072350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- 172.21.15.22:0/3340855063 >> 172.21.15.22:0/3340855063 conn(0x7f74280b21d0 msgr2=0x7f74280b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- 172.21.15.22:0/3340855063 shutdown_connections 2024-04-27T07:51:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- 172.21.15.22:0/3340855063 wait complete. 2024-04-27T07:51:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 Processor -- start 2024-04-27T07:51:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- start start 2024-04-27T07:51:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 0x7f742813f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7428073130 con 0x7f742806ec80 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7428072fb0 con 0x7f7428071f70 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742f888640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74280cceb0 con 0x7f74280cffe0 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742d624640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742d624640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40918/0 (socket says 172.21.15.22:40918) 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.683+0000 7f742d624640 1 -- 172.21.15.22:0/2138433679 learned_addr learned my addr 172.21.15.22:0/2138433679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742ce23640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 0x7f742813f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 -- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 msgr2=0x7f742815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742de25640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742815c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 -- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 msgr2=0x7f742813f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 0x7f742813f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 -- 172.21.15.22:0/2138433679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74281370e0 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742ce23640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 0x7f742813f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742d624640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74180ae040 tx=0x7f741801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180ba020 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74180b6620 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180b0040 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74281373c0 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7428137c60 con 0x7f742806ec80 2024-04-27T07:51:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7428137590 con 0x7f742806ec80 2024-04-27T07:51:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74180bd020 con 0x7f742806ec80 2024-04-27T07:51:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 0x7f740c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7418120c60 con 0x7f742806ec80 2024-04-27T07:51:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742ce23640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 0x7f740c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.687+0000 7f742ce23640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 0x7f740c06a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7428135eb0 tx=0x7f7424008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:03.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.691+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74180c4030 con 0x7f742806ec80 2024-04-27T07:51:03.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.855+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f742804ff10 con 0x7f742806ec80 2024-04-27T07:51:03.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.855+0000 7f741dffb640 1 -- 172.21.15.22:0/2138433679 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7418016110 con 0x7f742806ec80 2024-04-27T07:51:03.862 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 msgr2=0x7f740c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 0x7f740c06a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7428135eb0 tx=0x7f7424008040 comp rx=0 tx=0).stop 2024-04-27T07:51:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 msgr2=0x7f7428138160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74180ae040 tx=0x7f741801acd0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 shutdown_connections 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f740c068380 0x7f740c06a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74280cffe0 0x7f742815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7428071f70 0x7f742813f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 --2- 172.21.15.22:0/2138433679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f742806ec80 0x7f7428138160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 >> 172.21.15.22:0/2138433679 conn(0x7f74280b21d0 msgr2=0x7f742815de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 shutdown_connections 2024-04-27T07:51:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.859+0000 7f742f888640 1 -- 172.21.15.22:0/2138433679 wait complete. 2024-04-27T07:51:03.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.995+0000 7f9620bdb640 1 Processor -- start 2024-04-27T07:51:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.995+0000 7f9620bdb640 1 -- start start 2024-04-27T07:51:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 0x7f961c072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f961c145a00 0x7f961c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c06e050 con 0x7f961c072030 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f961c06f330 con 0x7f961c145a00 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f961c0ceae0 con 0x7f961c1454b0 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f961c145a00 0x7f961c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f961c145a00 0x7f961c06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40926/0 (socket says 172.21.15.22:40926) 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 -- 172.21.15.22:0/787011219 learned_addr learned my addr 172.21.15.22:0/787011219 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f961a59c640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 0x7f961c072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 -- 172.21.15.22:0/787011219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 msgr2=0x7f961c072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 0x7f961c072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 -- 172.21.15.22:0/787011219 >> v1:172.21.15.183:6789/0 conn(0x7f961c072030 legacy=0x7f961c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 -- 172.21.15.22:0/787011219 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c0cedc0 con 0x7f961c145a00 2024-04-27T07:51:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f961a59c640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 0x7f961c072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9619d9b640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c145a00 0x7f961c06edf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9610009870 tx=0x7f961001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e035442c143e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f961959a640 1 -- 172.21.15.22:0/787011219 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100b0070 con 0x7f961c145a00 2024-04-27T07:51:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f961959a640 1 -- 172.21.15.22:0/787011219 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96100ab600 con 0x7f961c145a00 2024-04-27T07:51:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f961959a640 1 -- 172.21.15.22:0/787011219 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100ab9d0 con 0x7f961c145a00 2024-04-27T07:51:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- 172.21.15.22:0/787011219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c145a00 msgr2=0x7f961c06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c145a00 0x7f961c06edf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9610009870 tx=0x7f961001c520 comp rx=0 tx=0).stop 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- 172.21.15.22:0/787011219 shutdown_connections 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c145a00 0x7f961c06edf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 --2- 172.21.15.22:0/787011219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c1454b0 0x7f961c072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- 172.21.15.22:0/787011219 >> 172.21.15.22:0/787011219 conn(0x7f961c067f10 msgr2=0x7f961c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- 172.21.15.22:0/787011219 shutdown_connections 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:03.999+0000 7f9620bdb640 1 -- 172.21.15.22:0/787011219 wait complete. 2024-04-27T07:51:04.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 Processor -- start 2024-04-27T07:51:04.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- start start 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c1454b0 0x7f961c13b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 0x7f961c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f961c15d820 con 0x7f961c072030 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f961c0728d0 con 0x7f961c1454b0 2024-04-27T07:51:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f961c15d9a0 con 0x7f961c145a00 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961ad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 0x7f961c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961ad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 0x7f961c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55476/0 (socket says 172.21.15.22:55476) 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40940/0 (socket says 172.21.15.22:40940) 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961ad9d640 1 -- 172.21.15.22:0/1074326769 learned_addr learned my addr 172.21.15.22:0/1074326769 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9619d9b640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c1454b0 0x7f961c13b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 -- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 msgr2=0x7f961c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 0x7f961c0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 -- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c1454b0 msgr2=0x7f961c13b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c1454b0 0x7f961c13b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 -- 172.21.15.22:0/1074326769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c13be10 con 0x7f961c072030 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f961a59c640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7f0 tx=0x7f960c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c015070 con 0x7f961c072030 2024-04-27T07:51:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c004220 con 0x7f961c072030 2024-04-27T07:51:04.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c003620 con 0x7f961c072030 2024-04-27T07:51:04.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c13c0b0 con 0x7f961c072030 2024-04-27T07:51:04.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c13c670 con 0x7f961c072030 2024-04-27T07:51:04.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.003+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c13cd90 con 0x7f961c072030 2024-04-27T07:51:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.007+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f960c0039d0 con 0x7f961c072030 2024-04-27T07:51:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.007+0000 7f960affd640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 0x7f95fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.007+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f960c086a60 con 0x7f961c072030 2024-04-27T07:51:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.007+0000 7f9619d9b640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 0x7f95fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.007+0000 7f9619d9b640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 0x7f95fc06a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96100ad000 tx=0x7f96100a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.011+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f960c0568a0 con 0x7f961c072030 2024-04-27T07:51:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.207+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c0b2bd0 con 0x7f961c072030 2024-04-27T07:51:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.207+0000 7f960affd640 1 -- 172.21.15.22:0/1074326769 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f960c05aba0 con 0x7f961c072030 2024-04-27T07:51:04.214 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:04.214 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:04.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 msgr2=0x7f95fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 0x7f95fc06a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96100ad000 tx=0x7f96100a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 msgr2=0x7f961c13e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7f0 tx=0x7f960c00ccb0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 shutdown_connections 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f95fc068380 0x7f95fc06a840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961c145a00 0x7f961c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c1454b0 0x7f961c13b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 --2- 172.21.15.22:0/1074326769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f961c072030 0x7f961c13e940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 >> 172.21.15.22:0/1074326769 conn(0x7f961c067f10 msgr2=0x7f961c142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 shutdown_connections 2024-04-27T07:51:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.211+0000 7f9620bdb640 1 -- 172.21.15.22:0/1074326769 wait complete. 2024-04-27T07:51:04.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.351+0000 7f5a9e84c640 1 Processor -- start 2024-04-27T07:51:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.351+0000 7f5a9e84c640 1 -- start start 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9e84c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a981454b0 0x7f5a98072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9e84c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 0x7f5a9806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9e84c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9806e050 con 0x7f5a98145a00 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9e84c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9806f330 con 0x7f5a981454b0 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9e84c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5a980ceae0 con 0x7f5a98072740 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a981454b0 0x7f5a98072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a981454b0 0x7f5a98072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40944/0 (socket says 172.21.15.22:40944) 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 -- 172.21.15.22:0/2813196263 learned_addr learned my addr 172.21.15.22:0/2813196263 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9cde9640 1 --2- 172.21.15.22:0/2813196263 >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 0x7f5a9806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 -- 172.21.15.22:0/2813196263 >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 msgr2=0x7f5a9806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 --2- 172.21.15.22:0/2813196263 >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 0x7f5a9806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 -- 172.21.15.22:0/2813196263 >> v1:172.21.15.183:6789/0 conn(0x7f5a98145a00 legacy=0x7f5a98072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 -- 172.21.15.22:0/2813196263 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a980cedc0 con 0x7f5a981454b0 2024-04-27T07:51:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a9cde9640 1 --2- 172.21.15.22:0/2813196263 >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 0x7f5a9806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:04.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a97fff640 1 --2- 172.21.15.22:0/2813196263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a981454b0 0x7f5a98072fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b3510 tx=0x7f5a8801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478924a85b67e089 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a96ffd640 1 -- 172.21.15.22:0/2813196263 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880b0070 con 0x7f5a981454b0 2024-04-27T07:51:04.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a96ffd640 1 -- 172.21.15.22:0/2813196263 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a880ab600 con 0x7f5a981454b0 2024-04-27T07:51:04.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.355+0000 7f5a96ffd640 1 -- 172.21.15.22:0/2813196263 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880ab9d0 con 0x7f5a981454b0 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/2813196263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a981454b0 msgr2=0x7f5a98072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/2813196263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a981454b0 0x7f5a98072fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b3510 tx=0x7f5a8801c4e0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/2813196263 shutdown_connections 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/2813196263 >> v2:172.21.15.183:3300/0 conn(0x7f5a98072740 0x7f5a9806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/2813196263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a981454b0 0x7f5a98072fa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/2813196263 >> 172.21.15.22:0/2813196263 conn(0x7f5a98067f10 msgr2=0x7f5a9806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/2813196263 shutdown_connections 2024-04-27T07:51:04.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/2813196263 wait complete. 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 Processor -- start 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- start start 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 0x7f5a980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a981454b0 0x7f5a98141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9815d690 con 0x7f5a98145a00 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9815d510 con 0x7f5a98072740 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9815d810 con 0x7f5a981454b0 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40958/0 (socket says 172.21.15.22:40958) 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 -- 172.21.15.22:0/1459337857 learned_addr learned my addr 172.21.15.22:0/1459337857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a97fff640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 0x7f5a980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a977fe640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a981454b0 0x7f5a98141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 -- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a981454b0 msgr2=0x7f5a98141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a981454b0 0x7f5a98141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 -- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 msgr2=0x7f5a980b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 0x7f5a980b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 -- 172.21.15.22:0/1459337857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a980b3020 con 0x7f5a98145a00 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a97fff640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 0x7f5a980b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9cde9640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ce80 tx=0x7f5a9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9001a070 con 0x7f5a98145a00 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9000d880 con 0x7f5a98145a00 2024-04-27T07:51:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a980b2780 con 0x7f5a98145a00 2024-04-27T07:51:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000dc70 con 0x7f5a98145a00 2024-04-27T07:51:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.359+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a980b2d40 con 0x7f5a98145a00 2024-04-27T07:51:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a98068c30 con 0x7f5a98145a00 2024-04-27T07:51:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5a9001d070 con 0x7f5a98145a00 2024-04-27T07:51:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a94ff9640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 0x7f5a8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5a90087910 con 0x7f5a98145a00 2024-04-27T07:51:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a97fff640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 0x7f5a8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.363+0000 7f5a97fff640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 0x7f5a8406a840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5a880ad000 tx=0x7f5a88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.367+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a90057860 con 0x7f5a98145a00 2024-04-27T07:51:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.531+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a981318a0 con 0x7f5a98145a00 2024-04-27T07:51:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.531+0000 7f5a94ff9640 1 -- 172.21.15.22:0/1459337857 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f5a9005bb60 con 0x7f5a98145a00 2024-04-27T07:51:04.538 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:04.538 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 msgr2=0x7f5a8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 0x7f5a8406a840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5a880ad000 tx=0x7f5a88008040 comp rx=0 tx=0).stop 2024-04-27T07:51:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 msgr2=0x7f5a98141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ce80 tx=0x7f5a9000a710 comp rx=0 tx=0).stop 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 shutdown_connections 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5a84068380 0x7f5a8406a840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a98145a00 0x7f5a98141590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a981454b0 0x7f5a98141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 --2- 172.21.15.22:0/1459337857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a98072740 0x7f5a980b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 >> 172.21.15.22:0/1459337857 conn(0x7f5a98067f10 msgr2=0x7f5a9806f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 shutdown_connections 2024-04-27T07:51:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.535+0000 7f5a9e84c640 1 -- 172.21.15.22:0/1459337857 wait complete. 2024-04-27T07:51:04.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:04.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:04.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.675+0000 7f7399b58640 1 Processor -- start 2024-04-27T07:51:04.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.675+0000 7f7399b58640 1 -- start start 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7399b58640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f739415c860 0x7f739415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7399b58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f73940ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7399b58640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73940b3ef0 con 0x7f73941604f0 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7399b58640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f739415dea0 con 0x7f739415c860 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7399b58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f739415d600 con 0x7f739415d180 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7393fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f73941604f0 0x7f73940b1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41256/0 (socket says 172.21.15.22:41256) 2024-04-27T07:51:04.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7393fff640 1 -- 172.21.15.22:0/1324826394 learned_addr learned my addr 172.21.15.22:0/1324826394 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f73937fe640 1 --2- 172.21.15.22:0/1324826394 >> v2:172.21.15.183:3300/0 conn(0x7f739415c860 0x7f739415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f738affd640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f73940ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327174913 0 0) 0x7f73940b3ef0 con 0x7f73941604f0 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7380003530 con 0x7f73941604f0 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f73937fe640 1 -- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 msgr2=0x7f73940ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f73937fe640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f73940ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f73937fe640 1 -- 172.21.15.22:0/1324826394 >> v1:172.21.15.22:6789/0 conn(0x7f73941604f0 legacy=0x7f73940b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 69713206 0 0) 0x7f7380003530 con 0x7f73941604f0 2024-04-27T07:51:04.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.679+0000 7f73937fe640 1 -- 172.21.15.22:0/1324826394 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739415d8e0 con 0x7f739415c860 2024-04-27T07:51:04.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f73937fe640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739415c860 0x7f739415cc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73840097c0 tx=0x7f738401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11efae90d59cfcf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840b0070 con 0x7f739415c860 2024-04-27T07:51:04.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73840ab600 con 0x7f739415c860 2024-04-27T07:51:04.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7392ffd640 1 -- 172.21.15.22:0/1324826394 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840ab9b0 con 0x7f739415c860 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739415c860 msgr2=0x7f739415cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739415c860 0x7f739415cc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73840097c0 tx=0x7f738401c3e0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- 172.21.15.22:0/1324826394 shutdown_connections 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f73940ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- 172.21.15.22:0/1324826394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739415c860 0x7f739415cc40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- 172.21.15.22:0/1324826394 >> 172.21.15.22:0/1324826394 conn(0x7f739409aa30 msgr2=0x7f73940b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- 172.21.15.22:0/1324826394 shutdown_connections 2024-04-27T07:51:04.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- 172.21.15.22:0/1324826394 wait complete. 2024-04-27T07:51:04.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 Processor -- start 2024-04-27T07:51:04.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- start start 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f739415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73941604f0 0x7f73940af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.683+0000 7f7399b58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7394169680 con 0x7f739415c860 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7399b58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7394169800 con 0x7f73941604f0 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7399b58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7394169980 con 0x7f739415d180 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40974/0 (socket says 172.21.15.22:40974) 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 -- 172.21.15.22:0/3821755082 learned_addr learned my addr 172.21.15.22:0/3821755082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:04.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7393fff640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73941604f0 0x7f73940af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f738affd640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f739415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 -- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 msgr2=0x7f739415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f739415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 -- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73941604f0 msgr2=0x7f73940af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73941604f0 0x7f73940af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 -- 172.21.15.22:0/3821755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7394156910 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f73937fe640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7384000c10 tx=0x7f7384019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840b9020 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73840ab7b0 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7394156bf0 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840b0040 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941571b0 con 0x7f739415c860 2024-04-27T07:51:04.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739404ff10 con 0x7f739415c860 2024-04-27T07:51:04.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73840bc020 con 0x7f739415c860 2024-04-27T07:51:04.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 0x7f737406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:04.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7384120810 con 0x7f739415c860 2024-04-27T07:51:04.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.687+0000 7f738affd640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 0x7f737406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:04.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.691+0000 7f738affd640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 0x7f737406a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73940cafb0 tx=0x7f737c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.691+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73840c3030 con 0x7f739415c860 2024-04-27T07:51:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.851+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f739415c090 con 0x7f739415c860 2024-04-27T07:51:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.851+0000 7f7390ff9640 1 -- 172.21.15.22:0/3821755082 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7384016090 con 0x7f739415c860 2024-04-27T07:51:04.858 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 msgr2=0x7f737406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 0x7f737406a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73940cafb0 tx=0x7f737c008040 comp rx=0 tx=0).stop 2024-04-27T07:51:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 msgr2=0x7f7394153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7384000c10 tx=0x7f7384019180 comp rx=0 tx=0).stop 2024-04-27T07:51:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 shutdown_connections 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7374068380 0x7f737406a840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73941604f0 0x7f73940af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f739415d180 0x7f739415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 --2- 172.21.15.22:0/3821755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f739415c860 0x7f7394153860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 >> 172.21.15.22:0/3821755082 conn(0x7f739409aa30 msgr2=0x7f73940b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 shutdown_connections 2024-04-27T07:51:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.855+0000 7f7399b58640 1 -- 172.21.15.22:0/3821755082 wait complete. 2024-04-27T07:51:04.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 Processor -- start 2024-04-27T07:51:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 -- start start 2024-04-27T07:51:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f640cffe0 0x7f7f64164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f641668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640cdb10 con 0x7f7f64071f70 2024-04-27T07:51:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7f640ce0d0 con 0x7f7f640cffe0 2024-04-27T07:51:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f69b97640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f640cd8c0 con 0x7f7f6406f750 2024-04-27T07:51:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f641668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f640cffe0 0x7f7f64164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f641668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:55510/0 (socket says 172.21.15.22:55510) 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f640cffe0 0x7f7f64164390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:55632/0 (socket says 172.21.15.22:55632) 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f637fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f7f64071f70 0x7f7f64072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:33948/0 (socket says 172.21.15.22:33948) 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f63fff640 1 -- 172.21.15.22:0/3485310369 learned_addr learned my addr 172.21.15.22:0/3485310369 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 -- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 msgr2=0x7f7f641668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 --2- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f641668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 -- 172.21.15.22:0/3485310369 >> v1:172.21.15.183:6789/0 conn(0x7f7f64071f70 legacy=0x7f7f64072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:04.999+0000 7f7f62ffd640 1 -- 172.21.15.22:0/3485310369 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640cddf0 con 0x7f7f640cffe0 2024-04-27T07:51:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f62ffd640 1 --2- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f64164390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7f580097c0 tx=0x7f7f5801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7362bb99608b8579 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f627fc640 1 -- 172.21.15.22:0/3485310369 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580b0070 con 0x7f7f640cffe0 2024-04-27T07:51:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f627fc640 1 -- 172.21.15.22:0/3485310369 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f580ab600 con 0x7f7f640cffe0 2024-04-27T07:51:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f627fc640 1 -- 172.21.15.22:0/3485310369 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580ab9d0 con 0x7f7f640cffe0 2024-04-27T07:51:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 msgr2=0x7f7f64164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 --2- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f64164390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7f580097c0 tx=0x7f7f5801c410 comp rx=0 tx=0).stop 2024-04-27T07:51:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- 172.21.15.22:0/3485310369 shutdown_connections 2024-04-27T07:51:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 --2- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f641668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 --2- 172.21.15.22:0/3485310369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f64164390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- 172.21.15.22:0/3485310369 >> 172.21.15.22:0/3485310369 conn(0x7f7f640b21d0 msgr2=0x7f7f640b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- 172.21.15.22:0/3485310369 shutdown_connections 2024-04-27T07:51:05.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- 172.21.15.22:0/3485310369 wait complete. 2024-04-27T07:51:05.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 Processor -- start 2024-04-27T07:51:05.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.003+0000 7f7f69b97640 1 -- start start 2024-04-27T07:51:05.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f640aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f6413e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640cd030 con 0x7f7f64071f70 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640cceb0 con 0x7f7f640cffe0 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f64169640 con 0x7f7f6406f750 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40982/0 (socket says 172.21.15.22:40982) 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 -- 172.21.15.22:0/2112813212 learned_addr learned my addr 172.21.15.22:0/2112813212 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f637fe640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f640aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f63fff640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f6413e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 -- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 msgr2=0x7f7f640aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f640aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 -- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 msgr2=0x7f7f6413e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f6413e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 -- 172.21.15.22:0/2112813212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6413efa0 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f62ffd640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f580ae040 tx=0x7f7f580191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580ba070 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f580b43f0 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6413f280 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580b0040 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6413f840 con 0x7f7f64071f70 2024-04-27T07:51:05.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6404ff10 con 0x7f7f64071f70 2024-04-27T07:51:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7f580bd070 con 0x7f7f64071f70 2024-04-27T07:51:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 0x7f7f5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7f5811f9d0 con 0x7f7f64071f70 2024-04-27T07:51:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.007+0000 7f7f637fe640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 0x7f7f5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.011+0000 7f7f637fe640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 0x7f7f5006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f54000f70 tx=0x7f7f54007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.011+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f58016090 con 0x7f7f64071f70 2024-04-27T07:51:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.171+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f64068f60 con 0x7f7f64071f70 2024-04-27T07:51:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f4bfff640 1 -- 172.21.15.22:0/2112813212 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7f580eff00 con 0x7f7f64071f70 2024-04-27T07:51:05.179 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 msgr2=0x7f7f5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 0x7f7f5006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f54000f70 tx=0x7f7f54007c90 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 msgr2=0x7f7f64140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f580ae040 tx=0x7f7f580191b0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 shutdown_connections 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f50068380 0x7f7f5006a840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f640cffe0 0x7f7f6413e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f64071f70 0x7f7f64140f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 --2- 172.21.15.22:0/2112813212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f6406f750 0x7f7f640aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 >> 172.21.15.22:0/2112813212 conn(0x7f7f640b21d0 msgr2=0x7f7f6415ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 shutdown_connections 2024-04-27T07:51:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.175+0000 7f7f69b97640 1 -- 172.21.15.22:0/2112813212 wait complete. 2024-04-27T07:51:05.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:05.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.327+0000 7f34e9f80640 1 Processor -- start 2024-04-27T07:51:05.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.327+0000 7f34e9f80640 1 -- start start 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.327+0000 7f34e9f80640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34e406f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.327+0000 7f34e9f80640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f34e4072f10 0x7f34e40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e9f80640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e415c2a0 con 0x7f34e40af990 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e9f80640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f34e415b4e0 con 0x7f34e4072f10 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e9f80640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f34e415b7c0 con 0x7f34e415af30 2024-04-27T07:51:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f34e40af990 0x7f34e4072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41264/0 (socket says 172.21.15.22:41264) 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 -- 172.21.15.22:0/225637147 learned_addr learned my addr 172.21.15.22:0/225637147 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e37fe640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34e406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e3fff640 1 --2- 172.21.15.22:0/225637147 >> v2:172.21.15.183:3300/0 conn(0x7f34e4072f10 0x7f34e40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421531354 0 0) 0x7f34e415c2a0 con 0x7f34e40af990 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34cc003530 con 0x7f34e40af990 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 497255191 0 0) 0x7f34cc003530 con 0x7f34e40af990 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 msgr2=0x7f34e406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34e406f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 >> v2:172.21.15.183:3300/0 conn(0x7f34e4072f10 msgr2=0x7f34e40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 --2- 172.21.15.22:0/225637147 >> v2:172.21.15.183:3300/0 conn(0x7f34e4072f10 0x7f34e40b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e415baa0 con 0x7f34e40af990 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f34d8001680 con 0x7f34e40af990 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 >> v1:172.21.15.22:6789/0 conn(0x7f34e40af990 legacy=0x7f34e4072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34cc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34cc004d50 0x7f34cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34e415b4e0 con 0x7f34cc003f40 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34e415b7c0 con 0x7f34cc004d50 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc00ba20 con 0x7f34e415af30 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e37fe640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34cc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 -- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 msgr2=0x7f34cc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e415af30 0x7f34cc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e3fff640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34cc004d50 0x7f34cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 -- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34cc004d50 msgr2=0x7f34cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34cc004d50 0x7f34cc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 -- 172.21.15.22:0/225637147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc003530 con 0x7f34cc003f40 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34cc004810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c270 tx=0x7f34d801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72538e5fce3f421 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8007d60 con 0x7f34cc003f40 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d80a9020 con 0x7f34cc003f40 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.331+0000 7f34e27fc640 1 -- 172.21.15.22:0/225637147 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8005ae0 con 0x7f34cc003f40 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/225637147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 msgr2=0x7f34cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 --2- 172.21.15.22:0/225637147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34cc004810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c270 tx=0x7f34d801ccc0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e3fff640 1 -- 172.21.15.22:0/225637147 reap_dead start 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/225637147 shutdown_connections 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/225637147 >> 172.21.15.22:0/225637147 conn(0x7f34e40b21d0 msgr2=0x7f34e40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/225637147 shutdown_connections 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/225637147 wait complete. 2024-04-27T07:51:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 Processor -- start 2024-04-27T07:51:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- start start 2024-04-27T07:51:05.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e4158100 0x7f34e4158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 0x7f34e4168970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34e40b3da0 con 0x7f34cc003f40 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34e40b3c20 con 0x7f34e41684f0 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f34e40b3f20 con 0x7f34e4158100 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40996/0 (socket says 172.21.15.22:40996) 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 -- 172.21.15.22:0/1438488469 learned_addr learned my addr 172.21.15.22:0/1438488469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e3fff640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 0x7f34e4168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 -- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e4158100 msgr2=0x7f34e4158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e4158100 0x7f34e4158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e4158100 0x7f34e4158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 -- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 msgr2=0x7f34e4168970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 0x7f34e4168970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 -- 172.21.15.22:0/1438488469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e41551c0 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e3fff640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 0x7f34e4168970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e37fe640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34d4011610 tx=0x7f34d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400f8f0 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d400fa90 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400fe40 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e4167a50 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.335+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e4155740 con 0x7f34cc003f40 2024-04-27T07:51:05.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e4155980 con 0x7f34cc003f40 2024-04-27T07:51:05.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f34d4015460 con 0x7f34cc003f40 2024-04-27T07:51:05.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34cbfff640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f34d4086e30 con 0x7f34cc003f40 2024-04-27T07:51:05.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.339+0000 7f34e2ffd640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 0x7f34d006a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34d80aa000 tx=0x7f34d8001ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.343+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34d4019030 con 0x7f34cc003f40 2024-04-27T07:51:05.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.503+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34e404ff10 con 0x7f34cc003f40 2024-04-27T07:51:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.503+0000 7f34cbfff640 1 -- 172.21.15.22:0/1438488469 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f34d4056e30 con 0x7f34cc003f40 2024-04-27T07:51:05.510 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 msgr2=0x7f34d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 0x7f34d006a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34d80aa000 tx=0x7f34d8001ac0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 msgr2=0x7f34e4155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34d4011610 tx=0x7f34d4011ad0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 shutdown_connections 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e41684f0 0x7f34e4168970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34e4158100 0x7f34e4158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 --2- 172.21.15.22:0/1438488469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34cc003f40 0x7f34e4155b90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 >> 172.21.15.22:0/1438488469 conn(0x7f34e40b21d0 msgr2=0x7f34e40cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 shutdown_connections 2024-04-27T07:51:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.507+0000 7f34e9f80640 1 -- 172.21.15.22:0/1438488469 wait complete. 2024-04-27T07:51:05.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 Processor -- start 2024-04-27T07:51:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 -- start start 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 0x7fecc806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fecc806f6a0 0x7fecc80cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fecc80cfc40 0x7fecc815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fecc806fb00 con 0x7fecc806f6a0 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fecc80720d0 con 0x7fecc80cfc40 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7feccdd9d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fecc8072370 con 0x7fecc8144dd0 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fecc806f6a0 0x7fecc80cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fecc806f6a0 0x7fecc80cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41006/0 (socket says 172.21.15.22:41006) 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 -- 172.21.15.22:0/337469542 learned_addr learned my addr 172.21.15.22:0/337469542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc7fff640 1 --2- 172.21.15.22:0/337469542 >> v2:172.21.15.183:3300/0 conn(0x7fecc80cfc40 0x7fecc815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc77fe640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 0x7fecc806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 -- 172.21.15.22:0/337469542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 msgr2=0x7fecc806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 0x7fecc806f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 -- 172.21.15.22:0/337469542 >> v2:172.21.15.183:3300/0 conn(0x7fecc80cfc40 msgr2=0x7fecc815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/337469542 >> v2:172.21.15.183:3300/0 conn(0x7fecc80cfc40 0x7fecc815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 -- 172.21.15.22:0/337469542 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc8072650 con 0x7fecc806f6a0 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc77fe640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 0x7fecc806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.643+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc80cf700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecc806cc60 tx=0x7fecbc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59293d35e03fb5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc67fc640 1 -- 172.21.15.22:0/337469542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0b1070 con 0x7fecc806f6a0 2024-04-27T07:51:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc67fc640 1 -- 172.21.15.22:0/337469542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc0ac630 con 0x7fecc806f6a0 2024-04-27T07:51:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc67fc640 1 -- 172.21.15.22:0/337469542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0a8050 con 0x7fecc806f6a0 2024-04-27T07:51:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/337469542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 msgr2=0x7fecc80cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc80cf700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecc806cc60 tx=0x7fecbc01c520 comp rx=0 tx=0).stop 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/337469542 shutdown_connections 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- 172.21.15.22:0/337469542 >> v2:172.21.15.183:3300/0 conn(0x7fecc80cfc40 0x7fecc815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc80cf700 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- 172.21.15.22:0/337469542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc8144dd0 0x7fecc806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/337469542 >> 172.21.15.22:0/337469542 conn(0x7fecc8067f10 msgr2=0x7fecc806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/337469542 shutdown_connections 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/337469542 wait complete. 2024-04-27T07:51:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 Processor -- start 2024-04-27T07:51:05.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- start start 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80cfc40 0x7fecc813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8144dd0 0x7fecc8159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fecc80d01d0 con 0x7fecc806f6a0 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecc80d0050 con 0x7fecc8144dd0 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fecc80b3dd0 con 0x7fecc80cfc40 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41014/0 (socket says 172.21.15.22:41014) 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 -- 172.21.15.22:0/554727192 learned_addr learned my addr 172.21.15.22:0/554727192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc7fff640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8144dd0 0x7fecc8159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80cfc40 0x7fecc813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 -- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80cfc40 msgr2=0x7fecc813b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80cfc40 0x7fecc813b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 -- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8144dd0 msgr2=0x7fecc8159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8144dd0 0x7fecc8159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 -- 172.21.15.22:0/554727192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc813be50 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecc77fe640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fecb800ce00 tx=0x7fecb800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800f890 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb800fa30 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc81425b0 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800fe40 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.647+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc8142b70 con 0x7fecc806f6a0 2024-04-27T07:51:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc8068c30 con 0x7fecc806f6a0 2024-04-27T07:51:05.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fecb8018400 con 0x7fecc806f6a0 2024-04-27T07:51:05.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7fecaffff640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 0x7fecb406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fecb8087500 con 0x7fecc806f6a0 2024-04-27T07:51:05.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 0x7fecb406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.651+0000 7fecc6ffd640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 0x7fecb406a890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ae000 tx=0x7fecbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.655+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecb8014030 con 0x7fecc806f6a0 2024-04-27T07:51:05.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.847+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc8159540 con 0x7fecc806f6a0 2024-04-27T07:51:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7fecaffff640 1 -- 172.21.15.22:0/554727192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fecb800ae20 con 0x7fecc806f6a0 2024-04-27T07:51:05.855 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:05.855 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:05.856 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:05.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 msgr2=0x7fecb406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 0x7fecb406a890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ae000 tx=0x7fecbc008040 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 msgr2=0x7fecc813aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fecb800ce00 tx=0x7fecb800a650 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 shutdown_connections 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fecb40683d0 0x7fecb406a890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8144dd0 0x7fecc8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80cfc40 0x7fecc813b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 --2- 172.21.15.22:0/554727192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc806f6a0 0x7fecc813aec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 >> 172.21.15.22:0/554727192 conn(0x7fecc8067f10 msgr2=0x7fecc80b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 shutdown_connections 2024-04-27T07:51:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.851+0000 7feccdd9d640 1 -- 172.21.15.22:0/554727192 wait complete. 2024-04-27T07:51:05.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:05.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.975+0000 7f02933c9640 1 Processor -- start 2024-04-27T07:51:05.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.975+0000 7f02933c9640 1 -- start start 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f02933c9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f028c09aa30 0x7f028c09ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f02933c9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0ccc10 0x7f028c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f02933c9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f028c0d1520 con 0x7f028c15ed30 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f02933c9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f028c09b3d0 con 0x7f028c09aa30 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f02933c9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f028c06d6f0 con 0x7f028c0ccc10 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f028c09aa30 0x7f028c09ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f028c09aa30 0x7f028c09ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41016/0 (socket says 172.21.15.22:41016) 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 -- 172.21.15.22:0/1979497726 learned_addr learned my addr 172.21.15.22:0/1979497726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0291966640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0ccc10 0x7f028c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 -- 172.21.15.22:0/1979497726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0ccc10 msgr2=0x7f028c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0ccc10 0x7f028c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 -- 172.21.15.22:0/1979497726 >> v1:172.21.15.183:6789/0 conn(0x7f028c15ed30 legacy=0x7f028c06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.979+0000 7f0290964640 1 -- 172.21.15.22:0/1979497726 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c06d940 con 0x7f028c09aa30 2024-04-27T07:51:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0290964640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c09aa30 0x7f028c09ae90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f027c009870 tx=0x7f027c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8caff8bcb754d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0283fff640 1 -- 172.21.15.22:0/1979497726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0b2070 con 0x7f028c09aa30 2024-04-27T07:51:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0283fff640 1 -- 172.21.15.22:0/1979497726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027c0a9070 con 0x7f028c09aa30 2024-04-27T07:51:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0283fff640 1 -- 172.21.15.22:0/1979497726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0ad880 con 0x7f028c09aa30 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1979497726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c09aa30 msgr2=0x7f028c09ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c09aa30 0x7f028c09ae90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f027c009870 tx=0x7f027c01c560 comp rx=0 tx=0).stop 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1979497726 shutdown_connections 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0ccc10 0x7f028c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- 172.21.15.22:0/1979497726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c09aa30 0x7f028c09ae90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1979497726 >> 172.21.15.22:0/1979497726 conn(0x7f028c0b40b0 msgr2=0x7f028c0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1979497726 shutdown_connections 2024-04-27T07:51:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1979497726 wait complete. 2024-04-27T07:51:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 Processor -- start 2024-04-27T07:51:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- start start 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 0x7f028c0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0ccc10 0x7f028c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f028c16bb40 con 0x7f028c15ed30 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f028c16b9c0 con 0x7f028c0ccc10 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f028c16bcc0 con 0x7f028c09aa30 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41018/0 (socket says 172.21.15.22:41018) 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 -- 172.21.15.22:0/1512871678 learned_addr learned my addr 172.21.15.22:0/1512871678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0290964640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0ccc10 0x7f028c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291165640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 0x7f028c0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 -- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 msgr2=0x7f028c0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 0x7f028c0b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 -- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0ccc10 msgr2=0x7f028c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0ccc10 0x7f028c0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 -- 172.21.15.22:0/1512871678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c15c4d0 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291165640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 0x7f028c0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0291966640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0284007e60 tx=0x7f028400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284014420 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0284019070 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284014820 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c155110 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.983+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1556d0 con 0x7f028c15ed30 2024-04-27T07:51:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c069340 con 0x7f028c15ed30 2024-04-27T07:51:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f02840149c0 con 0x7f028c15ed30 2024-04-27T07:51:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f0281ffb640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 0x7f027006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0284087300 con 0x7f028c15ed30 2024-04-27T07:51:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f0291165640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 0x7f027006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.987+0000 7f0291165640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 0x7f027006a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02880016f0 tx=0x7f0288009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:05.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:05.991+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0284057820 con 0x7f028c15ed30 2024-04-27T07:51:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.155+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f028c071fd0 con 0x7f028c15ed30 2024-04-27T07:51:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.155+0000 7f0281ffb640 1 -- 172.21.15.22:0/1512871678 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f0284057120 con 0x7f028c15ed30 2024-04-27T07:51:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:06.162 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 msgr2=0x7f027006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 0x7f027006a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02880016f0 tx=0x7f0288009040 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 msgr2=0x7f028c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0284007e60 tx=0x7f028400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 shutdown_connections 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0270068380 0x7f027006a840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f028c15ed30 0x7f028c15ca10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0ccc10 0x7f028c0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 --2- 172.21.15.22:0/1512871678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c09aa30 0x7f028c0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 >> 172.21.15.22:0/1512871678 conn(0x7f028c0b40b0 msgr2=0x7f028c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 shutdown_connections 2024-04-27T07:51:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.159+0000 7f02933c9640 1 -- 172.21.15.22:0/1512871678 wait complete. 2024-04-27T07:51:06.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 Processor -- start 2024-04-27T07:51:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 -- start start 2024-04-27T07:51:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c06f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f051c0cfe10 0x7f051c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0720d0 con 0x7f051c06f6a0 2024-04-27T07:51:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f051c06fb00 con 0x7f051c0cfe10 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f0522d3e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f051c072370 con 0x7f051c144dd0 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f05212db640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f051c0cfe10 0x7f051c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f05212db640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f051c0cfe10 0x7f051c0b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41024/0 (socket says 172.21.15.22:41024) 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.295+0000 7f05212db640 1 -- 172.21.15.22:0/3382242214 learned_addr learned my addr 172.21.15.22:0/3382242214 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0520ada640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f05212db640 1 -- 172.21.15.22:0/3382242214 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 msgr2=0x7f051c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f05212db640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c06f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f05212db640 1 -- 172.21.15.22:0/3382242214 >> v1:172.21.15.183:6789/0 conn(0x7f051c06f6a0 legacy=0x7f051c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f05212db640 1 -- 172.21.15.22:0/3382242214 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c072650 con 0x7f051c0cfe10 2024-04-27T07:51:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0520ada640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c06f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f05212db640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c0cfe10 0x7f051c0b3dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0514009870 tx=0x7f051401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd63680d9cff1419 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f051b7fe640 1 -- 172.21.15.22:0/3382242214 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05140b0070 con 0x7f051c0cfe10 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f051b7fe640 1 -- 172.21.15.22:0/3382242214 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05140ab600 con 0x7f051c0cfe10 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f051b7fe640 1 -- 172.21.15.22:0/3382242214 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05140ab9d0 con 0x7f051c0cfe10 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 -- 172.21.15.22:0/3382242214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c0cfe10 msgr2=0x7f051c0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c0cfe10 0x7f051c0b3dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0514009870 tx=0x7f051401c520 comp rx=0 tx=0).stop 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 -- 172.21.15.22:0/3382242214 shutdown_connections 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c0cfe10 0x7f051c0b3dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3382242214 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c06f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 -- 172.21.15.22:0/3382242214 >> 172.21.15.22:0/3382242214 conn(0x7f051c067f10 msgr2=0x7f051c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 -- 172.21.15.22:0/3382242214 shutdown_connections 2024-04-27T07:51:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.299+0000 7f0522d3e640 1 -- 172.21.15.22:0/3382242214 wait complete. 2024-04-27T07:51:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 Processor -- start 2024-04-27T07:51:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- start start 2024-04-27T07:51:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0cfe10 0x7f051c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c0aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0b4ca0 con 0x7f051c06f6a0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0b4b20 con 0x7f051c0cfe10 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f051c0b4e20 con 0x7f051c144dd0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41026/0 (socket says 172.21.15.22:41026) 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 -- 172.21.15.22:0/3878935172 learned_addr learned my addr 172.21.15.22:0/3878935172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f051bfff640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0cfe10 0x7f051c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f05212db640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c0aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 -- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 msgr2=0x7f051c0aa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c0aa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 -- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0cfe10 msgr2=0x7f051c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0cfe10 0x7f051c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 -- 172.21.15.22:0/3878935172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c15e750 con 0x7f051c06f6a0 2024-04-27T07:51:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f05212db640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c0aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0520ada640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f050c00ce30 tx=0x7f050c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00f890 con 0x7f051c06f6a0 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050c00fa30 con 0x7f051c06f6a0 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c0a9630 con 0x7f051c06f6a0 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00fe00 con 0x7f051c06f6a0 2024-04-27T07:51:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c15e390 con 0x7f051c06f6a0 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.303+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c068c30 con 0x7f051c06f6a0 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.307+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f050c018400 con 0x7f051c06f6a0 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.307+0000 7f05197fa640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 0x7f050806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.307+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f050c0876c0 con 0x7f051c06f6a0 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.307+0000 7f051bfff640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 0x7f050806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.307+0000 7f051bfff640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 0x7f050806a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f051c13a670 tx=0x7f0510009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.311+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f050c014030 con 0x7f051c06f6a0 2024-04-27T07:51:06.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.507+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f051c0a9cf0 con 0x7f051c06f6a0 2024-04-27T07:51:06.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f05197fa640 1 -- 172.21.15.22:0/3878935172 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f050c00ae20 con 0x7f051c06f6a0 2024-04-27T07:51:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:06.515 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:06.515 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 msgr2=0x7f050806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 0x7f050806a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f051c13a670 tx=0x7f0510009040 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 msgr2=0x7f051c0acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f050c00ce30 tx=0x7f050c00a650 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 shutdown_connections 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0508068380 0x7f050806a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c144dd0 0x7f051c0aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0cfe10 0x7f051c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 --2- 172.21.15.22:0/3878935172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f051c06f6a0 0x7f051c0acfd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 >> 172.21.15.22:0/3878935172 conn(0x7f051c067f10 msgr2=0x7f051c15c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 shutdown_connections 2024-04-27T07:51:06.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:06.511+0000 7f0522d3e640 1 -- 172.21.15.22:0/3878935172 wait complete. 2024-04-27T07:51:06.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-27T07:50:48.129218+0000', 'modified': '2024-04-27T07:50:51.161261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6837/1295572944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6835', 'nonce': 1295572944}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1295572944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-27T07:51:06.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-27T07:51:06.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-27T07:51:07.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 Processor -- start 2024-04-27T07:51:07.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 -- start start 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2a809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd2a815fa60 0x7fd2a806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a809bc00 con 0x7fd2a806dc30 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a80d1b60 con 0x7fd2a815fa60 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2af0f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a809b080 con 0x7fd2a8161dd0 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2ad68f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd2a806dc30 0x7fd2a815e1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41272/0 (socket says 172.21.15.22:41272) 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1392007954 learned_addr learned my addr 172.21.15.22:0/1392007954 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2a809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1392007954 >> v2:172.21.15.183:3300/0 conn(0x7fd2a815fa60 0x7fd2a806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.067+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900596774 0 0) 0x7fd2a809bc00 con 0x7fd2a806dc30 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd290003530 con 0x7fd2a806dc30 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2895228862 0 0) 0x7fd290003530 con 0x7fd2a806dc30 2024-04-27T07:51:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 msgr2=0x7fd2a809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2a809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 >> v2:172.21.15.183:3300/0 conn(0x7fd2a815fa60 msgr2=0x7fd2a806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1392007954 >> v2:172.21.15.183:3300/0 conn(0x7fd2a815fa60 0x7fd2a806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a809b360 con 0x7fd2a806dc30 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fd2a0001a00 con 0x7fd2a806dc30 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 >> v1:172.21.15.22:6789/0 conn(0x7fd2a806dc30 legacy=0x7fd2a815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2900039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd290004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd290004d50 0x7fd2900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a809b080 con 0x7fd290003f40 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000b9d0 con 0x7fd290004d50 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd29000bba0 con 0x7fd2a8161dd0 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd290004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2900039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd290004d50 0x7fd2900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 -- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 msgr2=0x7fd2900039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a8161dd0 0x7fd2900039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 -- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd290004d50 msgr2=0x7fd2900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd290004d50 0x7fd2900051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 -- 172.21.15.22:0/1392007954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290003530 con 0x7fd290003f40 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd290004810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd298009b10 tx=0x7fd29801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ac7daf5669989 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980b2070 con 0x7fd290003f40 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2980a5560 con 0x7fd290003f40 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2a77fe640 1 -- 172.21.15.22:0/1392007954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980096b0 con 0x7fd290003f40 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 -- 172.21.15.22:0/1392007954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 msgr2=0x7fd290004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/1392007954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd290004810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd298009b10 tx=0x7fd29801ca80 comp rx=0 tx=0).stop 2024-04-27T07:51:07.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1392007954 reap_dead start 2024-04-27T07:51:07.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 -- 172.21.15.22:0/1392007954 shutdown_connections 2024-04-27T07:51:07.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 -- 172.21.15.22:0/1392007954 >> 172.21.15.22:0/1392007954 conn(0x7fd2a80b4060 msgr2=0x7fd2a80b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 -- 172.21.15.22:0/1392007954 shutdown_connections 2024-04-27T07:51:07.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.071+0000 7fd2af0f2640 1 -- 172.21.15.22:0/1392007954 wait complete. 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 Processor -- start 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- start start 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 0x7fd2a80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a81faef0 0x7fd2a80707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a815ffc0 con 0x7fd290003f40 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a815fe40 con 0x7fd2a81faef0 2024-04-27T07:51:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a81621b0 con 0x7fd2a81faa70 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41040/0 (socket says 172.21.15.22:41040) 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 -- 172.21.15.22:0/51895601 learned_addr learned my addr 172.21.15.22:0/51895601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 0x7fd2a80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a81faef0 0x7fd2a80707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 -- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 msgr2=0x7fd2a80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 0x7fd2a80702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 -- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a81faef0 msgr2=0x7fd2a80707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a81faef0 0x7fd2a80707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 -- 172.21.15.22:0/51895601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a80b7ec0 con 0x7fd290003f40 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 0x7fd2a80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0115e0 tx=0x7fd29c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2a57fa640 1 -- 172.21.15.22:0/51895601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00f8f0 con 0x7fd290003f40 2024-04-27T07:51:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2a57fa640 1 -- 172.21.15.22:0/51895601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29c00fa90 con 0x7fd290003f40 2024-04-27T07:51:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a80b6bc0 con 0x7fd290003f40 2024-04-27T07:51:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2a57fa640 1 -- 172.21.15.22:0/51895601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00fe40 con 0x7fd290003f40 2024-04-27T07:51:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.075+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a80b7180 con 0x7fd290003f40 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a57fa640 1 -- 172.21.15.22:0/51895601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd29c015460 con 0x7fd290003f40 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a57fa640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 0x7fd29406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a57fa640 1 -- 172.21.15.22:0/51895601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd29c087250 con 0x7fd290003f40 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 0x7fd29406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 Processor -- start 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 -- start start 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 0x7fd29406a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2980af000 tx=0x7fd2980046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c793a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55693c5e4be0 con 0x55693c793fd0 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55693c57ebf0 con 0x55693c7e0690 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55693c5e0ba0 con 0x55693c7e0fb0 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41054/0 (socket says 172.21.15.22:41054) 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1867657208 learned_addr learned my addr 172.21.15.22:0/1867657208 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c793a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 msgr2=0x55693c793a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c793a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 msgr2=0x55693c7e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1867657208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693c68c630 con 0x55693c793fd0 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8073790 tx=0x7fd2a0007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffa0db1b34b7480 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd28e7fc640 1 -- 172.21.15.22:0/1867657208 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00133f0 con 0x55693c793fd0 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd28e7fc640 1 -- 172.21.15.22:0/1867657208 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0018070 con 0x55693c793fd0 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.079+0000 7fd28e7fc640 1 -- 172.21.15.22:0/1867657208 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00137d0 con 0x55693c793fd0 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1867657208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 msgr2=0x55693c7943b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8073790 tx=0x7fd2a0007420 comp rx=0 tx=0).stop 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1867657208 shutdown_connections 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c7943b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c793a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1867657208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1867657208 >> 172.21.15.22:0/1867657208 conn(0x55693c78da70 msgr2=0x55693c7c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1867657208 shutdown_connections 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1867657208 wait complete. 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- start start 2024-04-27T07:51:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 Processor -- start 2024-04-27T07:51:07.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55693c6a0d10 con 0x55693c7e0fb0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55693c58e3a0 con 0x55693c7e0690 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55693c5a81c0 con 0x55693c793fd0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 10 client.0 _ll_register_callbacks cb 0x55693c781730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41058/0 (socket says 172.21.15.22:41058) 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1862049269 learned_addr learned my addr 172.21.15.22:0/1862049269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 msgr2=0x55693c7e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 msgr2=0x55693c7e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 -- 172.21.15.22:0/1862049269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693c62a0d0 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007460 tx=0x7fd2a0001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8241d35fd6334942 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0013e60 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd2a001d600 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a0018070 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd2a002c050 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 10 client.4674 fetch_fsmap learned FSMap version 8 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55693c56ae80 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0018070 con 0x55693c7e0fb0 2024-04-27T07:51:07.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fd2a002f020 con 0x55693c7e0fb0 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 20 client.4674 populate_metadata read hostname 'smithi022' 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7c5410 0x55693c7c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 4 client.4674 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 --> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693c57dbc0 con 0x55693c7c5410 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7c5410 0x55693c7c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2877fe640 10 client.4674 ms_handle_connect on v2:172.21.15.183:6835/1295572944 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.083+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7c5410 0x55693c7c57f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7372b821166361 server_cookie=60dd196ba3ed072d in_seq=0 out_seq=0 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2877fe640 1 -- 172.21.15.22:0/1862049269 <== mds.0 v2:172.21.15.183:6835/1295572944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55693c57dbc0 con 0x55693c7c5410 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2877fe640 10 client.4674 handle_command_reply: tid=0 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 client.4674 shutdown 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 msgr2=0x55693c7c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007460 tx=0x7fd2a0001dd0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 shutdown_connections 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7c5410 0x55693c7c57f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7c4670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2877fe640 0 client.4674 ms_handle_reset on v2:172.21.15.183:6835/1295572944 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1862049269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 >> 172.21.15.22:0/1862049269 conn(0x55693c78da70 msgr2=0x55693c793190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 shutdown_connections 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 1 -- 172.21.15.22:0/1862049269 wait complete. 2024-04-27T07:51:07.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.087+0000 7fd2b2273000 20 client.4674 trim_cache size 0 max 16384 2024-04-27T07:51:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.103+0000 7fd2b2273000 1 Processor -- start 2024-04-27T07:51:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.103+0000 7fd2b2273000 1 -- start start 2024-04-27T07:51:07.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8203910 con 0x55693c7e0fb0 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8148e00 con 0x55693c7e0690 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a80b5e50 con 0x55693c793fd0 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41072/0 (socket says 172.21.15.22:41072) 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 -- 172.21.15.22:0/2769648046 learned_addr learned my addr 172.21.15.22:0/2769648046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 -- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 msgr2=0x55693c7f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 -- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 msgr2=0x55693c7f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 -- 172.21.15.22:0/2769648046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693c5acb30 con 0x55693c7e0fb0 2024-04-27T07:51:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ad68f640 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00013e0 tx=0x7fd2a001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfba852ade125149 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd28dffb640 1 -- 172.21.15.22:0/2769648046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0013e60 con 0x55693c7e0fb0 2024-04-27T07:51:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd28dffb640 1 -- 172.21.15.22:0/2769648046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0018040 con 0x55693c7e0fb0 2024-04-27T07:51:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd28dffb640 1 -- 172.21.15.22:0/2769648046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0013ab0 con 0x55693c7e0fb0 2024-04-27T07:51:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- 172.21.15.22:0/2769648046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 msgr2=0x55693c7f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00013e0 tx=0x7fd2a001f070 comp rx=0 tx=0).stop 2024-04-27T07:51:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- 172.21.15.22:0/2769648046 shutdown_connections 2024-04-27T07:51:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c7e0fb0 0x55693c7f7600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c7f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- 172.21.15.22:0/2769648046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c793fd0 0x55693c7f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- 172.21.15.22:0/2769648046 >> 172.21.15.22:0/2769648046 conn(0x55693c78da70 msgr2=0x55693c7ed840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- 172.21.15.22:0/2769648046 shutdown_connections 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- 172.21.15.22:0/2769648046 wait complete. 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- start start 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 Processor -- start 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c798d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c7e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55693c5e3ef0 con 0x55693c793fd0 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55693c5efd00 con 0x55693c7e0690 2024-04-27T07:51:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55693c5e9210 con 0x55693c7e0fb0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2b2273000 10 client.0 _ll_register_callbacks cb 0x7fd2a0001ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41082/0 (socket says 172.21.15.22:41082) 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 -- 172.21.15.22:0/1403302321 learned_addr learned my addr 172.21.15.22:0/1403302321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c798d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 -- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 msgr2=0x55693c7e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c7e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 -- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 msgr2=0x55693c798d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c798d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 -- 172.21.15.22:0/1403302321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693c665a10 con 0x55693c793fd0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2ace8e640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd29c057130 tx=0x7fd29c011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7d52001d33cac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.107+0000 7fd2877fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c05fd80 con 0x55693c793fd0 2024-04-27T07:51:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd29c053810 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a0018040 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd29c080020 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 10 client.4680 fetch_fsmap learned FSMap version 8 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55693c63ce30 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0018040 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fd29c05b160 con 0x55693c793fd0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 10 client.4680 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 20 client.4680 populate_metadata read hostname 'smithi022' 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7e4930 0x55693c7e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 4 client.4680 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 --> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] -- command(tid 0: {"prefix": "status"}) -- 0x55693c5f27f0 con 0x55693c7e4930 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7e4930 0x55693c7e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 10 client.4680 ms_handle_connect on v2:172.21.15.183:6835/1295572944 2024-04-27T07:51:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2a7fff640 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7e4930 0x55693c7e4d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd831ffd6149cdf4 server_cookie=471da0deb7e3ab5c in_seq=0 out_seq=0 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 1 -- 172.21.15.22:0/1403302321 <== mds.0 v2:172.21.15.183:6835/1295572944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55693c5f27f0 con 0x55693c7e4930 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 10 client.4680 handle_command_reply: tid=0 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 client.4680 shutdown 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 msgr2=0x55693c798830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd29c057130 tx=0x7fd29c011530 comp rx=0 tx=0).stop 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 shutdown_connections 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:6835/1295572944,v1:172.21.15.183:6837/1295572944] conn(0x55693c7e4930 0x55693c7e4d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55693c7e0fb0 0x55693c7e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693c7e0690 0x55693c798d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 --2- 172.21.15.22:0/1403302321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55693c793fd0 0x55693c798830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2877fe640 0 client.4680 ms_handle_reset on v2:172.21.15.183:6835/1295572944 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 >> 172.21.15.22:0/1403302321 conn(0x55693c78da70 msgr2=0x55693c7ee0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 shutdown_connections 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 1 -- 172.21.15.22:0/1403302321 wait complete. 2024-04-27T07:51:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2b2273000 20 client.4680 trim_cache size 0 max 16384 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "807b2bc3-0b23-4239-8346-8619bcda9f4e", 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4273, 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 18.328787956999999, 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 8, 2024-04-27T07:51:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 21, 2024-04-27T07:51:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 21, 2024-04-27T07:51:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 25.007800410000002 2024-04-27T07:51:07.123 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 msgr2=0x7fd29406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 0x7fd29406a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2980af000 tx=0x7fd2980046d0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 msgr2=0x7fd2a80b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0115e0 tx=0x7fd29c011aa0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 shutdown_connections 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd294068380 0x7fd29406a840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a81faef0 0x7fd2a80707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2a81faa70 0x7fd2a80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 --2- 172.21.15.22:0/51895601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd290003f40 0x7fd2a80b7980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 >> 172.21.15.22:0/51895601 conn(0x7fd2a80b4060 msgr2=0x7fd2a8163e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 shutdown_connections 2024-04-27T07:51:07.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.111+0000 7fd2af0f2640 1 -- 172.21.15.22:0/51895601 wait complete. 2024-04-27T07:51:07.134 INFO:teuthology.run_tasks:Running task sequential... 2024-04-27T07:51:07.143 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-27T07:51:07.143 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-27T07:51:07.143 INFO:teuthology.task.print:Enabling mgr modules 2024-04-27T07:51:07.143 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-27T07:51:07.171 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-27T07:51:07.171 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-27T07:51:07.171 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-27T07:51:07.171 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T07:51:07.171 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:07.171 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:07.172 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:07.172 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T07:51:07.172 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:07.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.172 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2024-04-27T07:51:07.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.180 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2024-04-27T07:51:07.237 INFO:teuthology.orchestra.run.smithi183.stderr:Cannot find device "ceph-brx" 2024-04-27T07:51:07.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:51:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.238 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-27T07:51:07.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-04-27T07:51:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-04-27T07:51:07.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:51:07.305 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-27T07:51:07.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:51:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.471+0000 7fa41ed97640 1 Processor -- start 2024-04-27T07:51:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41ed97640 1 -- start start 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41ed97640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa4181623a0 0x7fa418162800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41ed97640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa418145220 con 0x7fa418144e40 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41ed97640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41806f0e0 con 0x7fa41806f560 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41ed97640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa418162df0 con 0x7fa4181623a0 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41cb33640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa418144e40 0x7fa41806e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41284/0 (socket says 172.21.15.22:41284) 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa41cb33640 1 -- 172.21.15.22:0/1105503008 learned_addr learned my addr 172.21.15.22:0/1105503008 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa417fff640 1 --2- 172.21.15.22:0/1105503008 >> v2:172.21.15.22:3300/0 conn(0x7fa4181623a0 0x7fa418162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa417fff640 1 -- 172.21.15.22:0/1105503008 >> v1:172.21.15.183:6789/0 conn(0x7fa41806f560 legacy=0x7fa4180aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:07.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa417fff640 1 -- 172.21.15.22:0/1105503008 >> v1:172.21.15.22:6789/0 conn(0x7fa418144e40 legacy=0x7fa41806e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.475+0000 7fa417fff640 1 -- 172.21.15.22:0/1105503008 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181630d0 con 0x7fa4181623a0 2024-04-27T07:51:07.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa417fff640 1 --2- 172.21.15.22:0/1105503008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4181623a0 0x7fa418162800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa400009d90 tx=0x7fa40001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf236b27c04fab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa4177fe640 1 -- 172.21.15.22:0/1105503008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000b2070 con 0x7fa4181623a0 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa4177fe640 1 -- 172.21.15.22:0/1105503008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4000a5900 con 0x7fa4181623a0 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa4177fe640 1 -- 172.21.15.22:0/1105503008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000a5cd0 con 0x7fa4181623a0 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- 172.21.15.22:0/1105503008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4181623a0 msgr2=0x7fa418162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 --2- 172.21.15.22:0/1105503008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4181623a0 0x7fa418162800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa400009d90 tx=0x7fa40001c930 comp rx=0 tx=0).stop 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- 172.21.15.22:0/1105503008 shutdown_connections 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 --2- 172.21.15.22:0/1105503008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4181623a0 0x7fa418162800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- 172.21.15.22:0/1105503008 >> 172.21.15.22:0/1105503008 conn(0x7fa418067f10 msgr2=0x7fa41806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- 172.21.15.22:0/1105503008 shutdown_connections 2024-04-27T07:51:07.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- 172.21.15.22:0/1105503008 wait complete. 2024-04-27T07:51:07.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 Processor -- start 2024-04-27T07:51:07.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- start start 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 0x7fa418147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4181623a0 0x7fa4181615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa418072370 con 0x7fa41806f560 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4180721f0 con 0x7fa418144e40 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41ed97640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa41816cd50 con 0x7fa4181623a0 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41100/0 (socket says 172.21.15.22:41100) 2024-04-27T07:51:07.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 -- 172.21.15.22:0/506290438 learned_addr learned my addr 172.21.15.22:0/506290438 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa417fff640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 0x7fa418147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41d334640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4181623a0 0x7fa4181615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 -- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4181623a0 msgr2=0x7fa4181615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4181623a0 0x7fa4181615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 -- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 msgr2=0x7fa418147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 0x7fa418147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa41cb33640 1 -- 172.21.15.22:0/506290438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41813f200 con 0x7fa41806f560 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.479+0000 7fa417fff640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 0x7fa418147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa41cb33640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c420 tx=0x7fa40800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408017070 con 0x7fa41806f560 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408005cf0 con 0x7fa41806f560 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41813f4e0 con 0x7fa41806f560 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080072f0 con 0x7fa41806f560 2024-04-27T07:51:07.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41813faa0 con 0x7fa41806f560 2024-04-27T07:51:07.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa418068c30 con 0x7fa41806f560 2024-04-27T07:51:07.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa40801a070 con 0x7fa41806f560 2024-04-27T07:51:07.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 0x7fa3f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:07.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa408085e70 con 0x7fa41806f560 2024-04-27T07:51:07.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa417fff640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 0x7fa3f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:07.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.483+0000 7fa417fff640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 0x7fa3f806a840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4000af000 tx=0x7fa4000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:07.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.487+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa418145220 con 0x7fa41806f560 2024-04-27T07:51:07.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa418142f90 con 0x7fa41806f560 2024-04-27T07:51:07.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa4157fa640 1 -- 172.21.15.22:0/506290438 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa408056390 con 0x7fa41806f560 2024-04-27T07:51:07.692 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:51:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 msgr2=0x7fa3f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 0x7fa3f806a840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4000af000 tx=0x7fa4000a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 msgr2=0x7fa41815b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c420 tx=0x7fa40800c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 shutdown_connections 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa3f8068380 0x7fa3f806a840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4181623a0 0x7fa4181615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418144e40 0x7fa418147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 --2- 172.21.15.22:0/506290438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41806f560 0x7fa41815b170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 >> 172.21.15.22:0/506290438 conn(0x7fa418067f10 msgr2=0x7fa4181431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 shutdown_connections 2024-04-27T07:51:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07.687+0000 7fa41ed97640 1 -- 172.21.15.22:0/506290438 wait complete. 2024-04-27T07:51:07.710 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:51:07.711 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:51:07.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.711 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-04-27T07:51:07.717 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-04-27T07:51:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:ba:2b:46:d0 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:51:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:07.719 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-27T07:51:07.719 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:07.719 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:07.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-04-27T07:51:07.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-04-27T07:51:07.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-04-27T07:51:07.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-27T07:51:07.720 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:07.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:07.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:07.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.844 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:51:07.853 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-04-27T07:51:07.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:07.855 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-27T07:51:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-27T07:51:07.947 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:07.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:08.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:08.025 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-27T07:51:08.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:08.071 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-04-27T07:51:08.119 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:08.119 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:08.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:08.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-27T07:51:08.119 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:08.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:08.237 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-27T07:51:08.237 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:08.237 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-27T07:51:08.238 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:08.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:08.348 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:08.348 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:08.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-04-27T07:51:08.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-04-27T07:51:08.348 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:08.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:08.452 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-27T07:51:08.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:08.452 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:08.498 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:51:08.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:08.499 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:08.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:08.606 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:08.654 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:08.655 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:08.656 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:08.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:08.657 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:08.703 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T07:51:08.704 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-27T07:51:08.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:08.763 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:08.816 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:08.817 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:08.818 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:08.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:08.819 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:08.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-04-27T07:51:08.823+0000 7fd686ab5580 -1 init, newargv = 0x55ea8cf3df70 newargc=15 2024-04-27T07:51:08.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[33912]: starting ceph client 2024-04-27T07:51:08.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[33912]: starting fuse 2024-04-27T07:51:09.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:09.949 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:09.999 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:10.000 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:10.000 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:10.000 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:10.000 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:10.000 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:10.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.001 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:10.048 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-04-27T07:51:10.048 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-27T07:51:10.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:51:10.048 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-04-27T07:51:10.049 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-04-27T07:51:10.050 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-04-27T07:51:10.174 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.33912.asok 2024-04-27T07:51:10.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:10.187 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33912.asok 2024-04-27T07:51:10.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33912.asok status 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "33912", 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:10.351 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4695, 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4695 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 992083630 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:51:10.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 992083630 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4695 192.168.0.1:0/992083630", 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/992083630", 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 8, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 21, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-04-27T07:51:10.353 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:51:10.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:51:10.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 Processor -- start 2024-04-27T07:51:10.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 -- start start 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 0x7f723c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f723c0b7930 0x7f723c0d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c0b3d30 con 0x7f723c0d4790 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f723c0d4540 con 0x7f723c0b7930 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7240b3d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f723c0ba7a0 con 0x7f723c15ebe0 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f723c0b7930 0x7f723c0d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f723c0b7930 0x7f723c0d27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45966/0 (socket says 172.21.15.22:45966) 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 -- 172.21.15.22:0/2890317580 learned_addr learned my addr 172.21.15.22:0/2890317580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7239d9b640 1 --2- 172.21.15.22:0/2890317580 >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 0x7f723c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 -- 172.21.15.22:0/2890317580 >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 msgr2=0x7f723c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 --2- 172.21.15.22:0/2890317580 >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 0x7f723c0cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 -- 172.21.15.22:0/2890317580 >> v1:172.21.15.183:6789/0 conn(0x7f723c0d4790 legacy=0x7f723c0d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f723ad9d640 1 -- 172.21.15.22:0/2890317580 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0ba4c0 con 0x7f723c0b7930 2024-04-27T07:51:10.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.487+0000 7f7239d9b640 1 --2- 172.21.15.22:0/2890317580 >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 0x7f723c0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:10.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f723ad9d640 1 --2- 172.21.15.22:0/2890317580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0b7930 0x7f723c0d27b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f723c0d0b70 tx=0x7f723401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a0da645840fae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:10.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f723959a640 1 -- 172.21.15.22:0/2890317580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340ab660 con 0x7f723c0b7930 2024-04-27T07:51:10.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f723959a640 1 -- 172.21.15.22:0/2890317580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72340b0070 con 0x7f723c0b7930 2024-04-27T07:51:10.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f723959a640 1 -- 172.21.15.22:0/2890317580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340aba20 con 0x7f723c0b7930 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- 172.21.15.22:0/2890317580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0b7930 msgr2=0x7f723c0d27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- 172.21.15.22:0/2890317580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0b7930 0x7f723c0d27b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f723c0d0b70 tx=0x7f723401c460 comp rx=0 tx=0).stop 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- 172.21.15.22:0/2890317580 shutdown_connections 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- 172.21.15.22:0/2890317580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0b7930 0x7f723c0d27b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- 172.21.15.22:0/2890317580 >> v2:172.21.15.183:3300/0 conn(0x7f723c15ebe0 0x7f723c0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- 172.21.15.22:0/2890317580 >> 172.21.15.22:0/2890317580 conn(0x7f723c067d10 msgr2=0x7f723c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- 172.21.15.22:0/2890317580 shutdown_connections 2024-04-27T07:51:10.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- 172.21.15.22:0/2890317580 wait complete. 2024-04-27T07:51:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 Processor -- start 2024-04-27T07:51:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- start start 2024-04-27T07:51:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f723c0b7930 0x7f723c149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f723c150680 0x7f723c0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0cff20 con 0x7f723c0d4790 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0cfda0 con 0x7f723c0b7930 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7240b3d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0d3500 con 0x7f723c150680 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45976/0 (socket says 172.21.15.22:45976) 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.491+0000 7f7239d9b640 1 -- 172.21.15.22:0/4230994161 learned_addr learned my addr 172.21.15.22:0/4230994161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f723a59c640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f723c0b7930 0x7f723c149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f723ad9d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f723c150680 0x7f723c0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 -- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f723c150680 msgr2=0x7f723c0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f723c150680 0x7f723c0cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 -- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f723c0b7930 msgr2=0x7f723c149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f723c0b7930 0x7f723c149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 -- 172.21.15.22:0/4230994161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c155330 con 0x7f723c0d4790 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7239d9b640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f723000c820 tx=0x7f723000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230015070 con 0x7f723c0d4790 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300079d0 con 0x7f723c0d4790 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c155050 con 0x7f723c0d4790 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230007d80 con 0x7f723c0d4790 2024-04-27T07:51:10.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c0cccf0 con 0x7f723c0d4790 2024-04-27T07:51:10.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c068c30 con 0x7f723c0d4790 2024-04-27T07:51:10.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7230018070 con 0x7f723c0d4790 2024-04-27T07:51:10.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 0x7f721c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:10.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7230086880 con 0x7f723c0d4790 2024-04-27T07:51:10.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f723a59c640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 0x7f721c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:10.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.495+0000 7f723a59c640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 0x7f721c06a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f722c001d50 tx=0x7f722c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.499+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7230056db0 con 0x7f723c0d4790 2024-04-27T07:51:10.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.699+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f723c0ceca0 con 0x7f723c0d4790 2024-04-27T07:51:10.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.699+0000 7f722affd640 1 -- 172.21.15.22:0/4230994161 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f72300566b0 con 0x7f723c0d4790 2024-04-27T07:51:10.706 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 msgr2=0x7f721c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 0x7f721c06a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f722c001d50 tx=0x7f722c009040 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 msgr2=0x7f723c149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f723000c820 tx=0x7f723000cce0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 shutdown_connections 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f721c068380 0x7f721c06a840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f723c150680 0x7f723c0cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f723c0d4790 0x7f723c149a60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 --2- 172.21.15.22:0/4230994161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f723c0b7930 0x7f723c149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 >> 172.21.15.22:0/4230994161 conn(0x7f723c067d10 msgr2=0x7f723c0b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 shutdown_connections 2024-04-27T07:51:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10.703+0000 7f7240b3d640 1 -- 172.21.15.22:0/4230994161 wait complete. 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:51:10.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:51:10.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:51:10.726 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-27T07:51:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.726 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:10.732 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42518sec preferred_lft 42518sec 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-04-27T07:51:10.733 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:ba:2b:46:d0 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-27T07:51:10.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:10.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.735 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:51:10.783 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-04-27T07:51:10.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.784 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-27T07:51:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:51:10.837 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:10.837 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:10.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-27T07:51:10.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-27T07:51:10.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-27T07:51:10.838 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:10.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:10.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.960 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-27T07:51:10.966 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T07:51:10.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:10.966 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-04-27T07:51:11.011 INFO:teuthology.orchestra.run.smithi022.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-27T07:51:11.012 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:11.012 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:11.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:11.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-27T07:51:11.012 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:11.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:11.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:11.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:11.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-27T07:51:11.148 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 9e:73:f7:25:11:7a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c73:f7ff:fe25:117a/64 scope link 2024-04-27T07:51:11.149 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:11.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-27T07:51:11.207 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 9e:73:f7:25:11:7a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c73:f7ff:fe25:117a/64 scope link 2024-04-27T07:51:11.208 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:11.209 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-27T07:51:11.210 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:11.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:11.351 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:11.351 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:51:11.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.1 up 2024-04-27T07:51:11.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.1 master ceph-brx 2024-04-27T07:51:11.351 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:51:11.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:11.409 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-27T07:51:11.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:11.410 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:11.454 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-27T07:51:11.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:11.455 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:11.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:11.560 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:11.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:11.611 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-04-27T07:51:11.663 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-27T07:51:11.664 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-27T07:51:11.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:11.722 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:11.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:11.774 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:11.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:2024-04-27T07:51:11.783+0000 7f5055ee7580 -1 init, newargv = 0x56214070adb0 newargc=15 2024-04-27T07:51:11.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[34057]: starting ceph client 2024-04-27T07:51:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-04-27T07:51:11.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[34057]: starting fuse 2024-04-27T07:51:12.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:51:12.910 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:51:12.962 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:51:12.963 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:51:12.964 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:12.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:12.965 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:51:13.011 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-04-27T07:51:13.012 INFO:teuthology.orchestra.run.smithi022.stdout:46 2024-04-27T07:51:13.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-27T07:51:13.012 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:51:13.013 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:51:13.014 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.2")) 2024-04-27T07:51:13.014 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-04-27T07:51:13.139 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.2.34057.asok 2024-04-27T07:51:13.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:13.153 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34057.asok 2024-04-27T07:51:13.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:13.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34057.asok status 2024-04-27T07:51:13.305 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "2", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "34057", 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4707, 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-04-27T07:51:13.306 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4707 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1902081810 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1902081810 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:13.307 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4707 192.168.0.2:0/1902081810", 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.2:0/1902081810", 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 8, 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 21, 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-04-27T07:51:13.308 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:51:13.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:51:13.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 Processor -- start 2024-04-27T07:51:13.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 -- start start 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 0x7f785006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7850145220 con 0x7f785006ef10 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7850162df0 con 0x7f7850144e40 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f78553eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f78501630d0 con 0x7f785006f560 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 0x7f785006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784e7fc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f785006ef10 0x7f78501623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:52830/0 (socket says 172.21.15.22:52830) 2024-04-27T07:51:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784e7fc640 1 -- 172.21.15.22:0/2748292230 learned_addr learned my addr 172.21.15.22:0/2748292230 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784f7fe640 1 -- 172.21.15.22:0/2748292230 >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 msgr2=0x7f785006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784f7fe640 1 --2- 172.21.15.22:0/2748292230 >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 0x7f785006e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784f7fe640 1 -- 172.21.15.22:0/2748292230 >> v1:172.21.15.183:6789/0 conn(0x7f785006ef10 legacy=0x7f78501623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784dffb640 1 -- 172.21.15.22:0/2748292230 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336251938 0 0) 0x7f7838001120 con 0x7f785006ef10 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784f7fe640 1 -- 172.21.15.22:0/2748292230 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785006e1d0 con 0x7f785006f560 2024-04-27T07:51:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.439+0000 7f784effd640 1 --2- 172.21.15.22:0/2748292230 >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 0x7f785006e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f784f7fe640 1 --2- 172.21.15.22:0/2748292230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785006f9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7848009d90 tx=0x7f784801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e522ace87a84b1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f784dffb640 1 -- 172.21.15.22:0/2748292230 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480b2070 con 0x7f785006f560 2024-04-27T07:51:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f784dffb640 1 -- 172.21.15.22:0/2748292230 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78480a5940 con 0x7f785006f560 2024-04-27T07:51:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f784dffb640 1 -- 172.21.15.22:0/2748292230 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480a5d10 con 0x7f785006f560 2024-04-27T07:51:13.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 -- 172.21.15.22:0/2748292230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 msgr2=0x7f785006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 --2- 172.21.15.22:0/2748292230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785006f9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7848009d90 tx=0x7f784801c970 comp rx=0 tx=0).stop 2024-04-27T07:51:13.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 -- 172.21.15.22:0/2748292230 shutdown_connections 2024-04-27T07:51:13.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 --2- 172.21.15.22:0/2748292230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785006f9e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 --2- 172.21.15.22:0/2748292230 >> v2:172.21.15.183:3300/0 conn(0x7f7850144e40 0x7f785006e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 -- 172.21.15.22:0/2748292230 >> 172.21.15.22:0/2748292230 conn(0x7f7850067f10 msgr2=0x7f785006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 -- 172.21.15.22:0/2748292230 shutdown_connections 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.443+0000 7f78553eb640 1 -- 172.21.15.22:0/2748292230 wait complete. 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 Processor -- start 2024-04-27T07:51:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- start start 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785013e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 0x7f78500ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78500af5e0 con 0x7f7850144e40 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78500af460 con 0x7f785015cdb0 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f785013ec90 con 0x7f785006f560 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45978/0 (socket says 172.21.15.22:45978) 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 -- 172.21.15.22:0/1960298409 learned_addr learned my addr 172.21.15.22:0/1960298409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784f7fe640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 0x7f78500ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784effd640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785013e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 -- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 msgr2=0x7f785013e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785013e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 -- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 msgr2=0x7f78500ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 0x7f78500ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 -- 172.21.15.22:0/1960298409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785015d320 con 0x7f7850144e40 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784f7fe640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 0x7f78500ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784effd640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785013e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f784e7fc640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f783800c7b0 tx=0x7f783800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838018070 con 0x7f7850144e40 2024-04-27T07:51:13.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78380072f0 con 0x7f7850144e40 2024-04-27T07:51:13.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78380047f0 con 0x7f7850144e40 2024-04-27T07:51:13.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785015d600 con 0x7f7850144e40 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.447+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785015dbc0 con 0x7f7850144e40 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7850068c30 con 0x7f7850144e40 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7838004990 con 0x7f7850144e40 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f78477fe640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 0x7f783006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7838085e40 con 0x7f7850144e40 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f784effd640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 0x7f783006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.451+0000 7f784effd640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 0x7f783006a840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7840009b10 tx=0x7f7840008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:13.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.455+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7850145220 con 0x7f7850144e40 2024-04-27T07:51:13.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.651+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7850160e10 con 0x7f7850144e40 2024-04-27T07:51:13.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.651+0000 7f78477fe640 1 -- 172.21.15.22:0/1960298409 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f7838056360 con 0x7f7850144e40 2024-04-27T07:51:13.657 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:51:13.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 msgr2=0x7f783006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 0x7f783006a840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7840009b10 tx=0x7f7840008040 comp rx=0 tx=0).stop 2024-04-27T07:51:13.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 msgr2=0x7f785015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:13.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f783800c7b0 tx=0x7f783800cc70 comp rx=0 tx=0).stop 2024-04-27T07:51:13.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 shutdown_connections 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7830068380 0x7f783006a840 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785015cdb0 0x7f78500ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7850144e40 0x7f785015c550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 --2- 172.21.15.22:0/1960298409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f785006f560 0x7f785013e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 >> 172.21.15.22:0/1960298409 conn(0x7f7850067f10 msgr2=0x7f785015fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 shutdown_connections 2024-04-27T07:51:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:13.655+0000 7f78553eb640 1 -- 172.21.15.22:0/1960298409 wait complete. 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:13.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:51:13.676 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:51:13.676 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:51:13.676 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-27T07:51:13.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:13.676 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f0 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f1 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:13.683 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42517sec preferred_lft 42517sec 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::ec4:7aff:fe88:8085/64 scope link 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 02:42:99:0d:61:5b brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:51:13.684 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:13.684 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-27T07:51:13.684 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add name ceph-brx type bridge 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr flush dev ceph-brx 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set ceph-brx up 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-27T07:51:13.685 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:13.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:13.805 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:13.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:13.812 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:51:13.821 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-04-27T07:51:13.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:13.822 DEBUG:teuthology.orchestra.run.smithi183:> route 2024-04-27T07:51:13.965 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-27T07:51:13.967 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:14.013 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:14.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:14.049 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2024-04-27T07:51:14.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:14.055 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2024-04-27T07:51:14.101 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:14.101 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:14.101 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:14.101 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-27T07:51:14.101 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:14.188 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:14.210 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:14.217 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-27T07:51:14.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-27T07:51:14.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-27T07:51:14.218 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:14.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:14.364 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:14.371 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:14.371 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:14.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2024-04-27T07:51:14.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.0 master ceph-brx 2024-04-27T07:51:14.371 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:14.451 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:14.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:14.472 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-27T07:51:14.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:14.472 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:14.517 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-27T07:51:14.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:14.518 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:14.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:14.622 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:14.670 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:14.670 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:14.670 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:14.671 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:14.672 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:14.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:14.673 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:14.721 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T07:51:14.722 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-27T07:51:14.766 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:14.782 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:14.829 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:14.830 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:14.831 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:14.832 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:14.832 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:14.833 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:14.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:2024-04-27T07:51:14.843+0000 7f18d8029580 -1 init, newargv = 0x55875bac0d50 newargc=15 2024-04-27T07:51:14.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[33296]: starting ceph client 2024-04-27T07:51:14.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[33296]: starting fuse 2024-04-27T07:51:15.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:15.971 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:16.018 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:16.018 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:16.018 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:16.018 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:16.020 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:16.021 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:16.021 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:16.021 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:16.021 INFO:teuthology.orchestra.run.smithi183.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:16.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.022 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:16.073 INFO:teuthology.orchestra.run.smithi183.stdout:45 2024-04-27T07:51:16.073 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> import glob 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> import re 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> import os 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> import subprocess 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> def _find_admin_socket(client_name): 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> files = glob.glob(asok_path) 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:16.074 DEBUG:teuthology.orchestra.run.smithi183:> # Given a non-glob path, it better be there 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> if "*" not in asok_path: 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> assert(len(files) == 1) 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> return files[0] 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> for f in files: 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> contents = proc_f.read() 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> if mountpoint in contents: 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> return f 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:51:16.075 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:16.076 DEBUG:teuthology.orchestra.run.smithi183:> print(_find_admin_socket("client.1")) 2024-04-27T07:51:16.076 DEBUG:teuthology.orchestra.run.smithi183:> ' 2024-04-27T07:51:16.203 INFO:teuthology.orchestra.run.smithi183.stdout:/var/run/ceph/ceph-client.1.33296.asok 2024-04-27T07:51:16.207 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:16.215 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33296.asok 2024-04-27T07:51:16.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.215 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33296.asok status 2024-04-27T07:51:16.365 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "metadata": { 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "entity_id": "1", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "hostname": "smithi183", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "pid": "33296", 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "root": "/" 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_count": 0, 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_pinned_count": 0, 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "id": 4609, 2024-04-27T07:51:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "inst": { 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "name": { 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "client", 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "num": 4609 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 290434363 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 290434363 2024-04-27T07:51:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "inst_str": "client.4609 192.168.0.1:0/290434363", 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "addr_str": "192.168.0.1:0/290434363", 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "inode_count": 1, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_epoch": 8, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch": 21, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "blocklisted": false, 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "fs_name": "cephfs" 2024-04-27T07:51:16.368 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-27T07:51:16.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:51:16.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 Processor -- start 2024-04-27T07:51:16.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 -- start start 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f73b0162da0 0x7f73b006e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f73b006fc70 0x7f73b00b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b006e630 con 0x7f73b00ce260 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f73b0163180 con 0x7f73b006fc70 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73b4e90640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f73b006f1b0 con 0x7f73b0162da0 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73aed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f73b006fc70 0x7f73b00b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73add9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f73b00ce260 0x7f73b006f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54878/0 (socket says 172.21.15.22:54878) 2024-04-27T07:51:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73add9b640 1 -- 172.21.15.22:0/2096050429 learned_addr learned my addr 172.21.15.22:0/2096050429 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73ae59c640 1 --2- 172.21.15.22:0/2096050429 >> v2:172.21.15.183:3300/0 conn(0x7f73b0162da0 0x7f73b006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73ad59a640 1 -- 172.21.15.22:0/2096050429 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255910745 0 0) 0x7f73b006e630 con 0x7f73b00ce260 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73ad59a640 1 -- 172.21.15.22:0/2096050429 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7398003530 con 0x7f73b00ce260 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73aed9d640 1 -- 172.21.15.22:0/2096050429 >> v2:172.21.15.183:3300/0 conn(0x7f73b0162da0 msgr2=0x7f73b006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73aed9d640 1 --2- 172.21.15.22:0/2096050429 >> v2:172.21.15.183:3300/0 conn(0x7f73b0162da0 0x7f73b006e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73aed9d640 1 -- 172.21.15.22:0/2096050429 >> v1:172.21.15.22:6789/0 conn(0x7f73b00ce260 legacy=0x7f73b006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.511+0000 7f73aed9d640 1 -- 172.21.15.22:0/2096050429 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b006e910 con 0x7f73b006fc70 2024-04-27T07:51:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73aed9d640 1 --2- 172.21.15.22:0/2096050429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b006fc70 0x7f73b00b3e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f73b00732e0 tx=0x7f73a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6428c001d28f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73ad59a640 1 -- 172.21.15.22:0/2096050429 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80b2070 con 0x7f73b006fc70 2024-04-27T07:51:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73ad59a640 1 -- 172.21.15.22:0/2096050429 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a80a57f0 con 0x7f73b006fc70 2024-04-27T07:51:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73ad59a640 1 -- 172.21.15.22:0/2096050429 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80a5bc0 con 0x7f73b006fc70 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- 172.21.15.22:0/2096050429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b006fc70 msgr2=0x7f73b00b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 --2- 172.21.15.22:0/2096050429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b006fc70 0x7f73b00b3e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f73b00732e0 tx=0x7f73a801c930 comp rx=0 tx=0).stop 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- 172.21.15.22:0/2096050429 shutdown_connections 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 --2- 172.21.15.22:0/2096050429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b006fc70 0x7f73b00b3e70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 --2- 172.21.15.22:0/2096050429 >> v2:172.21.15.183:3300/0 conn(0x7f73b0162da0 0x7f73b006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- 172.21.15.22:0/2096050429 >> 172.21.15.22:0/2096050429 conn(0x7f73b0067f10 msgr2=0x7f73b006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- 172.21.15.22:0/2096050429 shutdown_connections 2024-04-27T07:51:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- 172.21.15.22:0/2096050429 wait complete. 2024-04-27T07:51:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 Processor -- start 2024-04-27T07:51:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.515+0000 7f73b4e90640 1 -- start start 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b006fc70 0x7f73b0165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73b0162da0 0x7f73b01fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00b5700 con 0x7f73b00ce260 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00b5580 con 0x7f73b006fc70 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00b5880 con 0x7f73b0162da0 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45994/0 (socket says 172.21.15.22:45994) 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 -- 172.21.15.22:0/3022589216 learned_addr learned my addr 172.21.15.22:0/3022589216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73ae59c640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b006fc70 0x7f73b0165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 -- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73b0162da0 msgr2=0x7f73b01fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73b0162da0 0x7f73b01fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 -- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b006fc70 msgr2=0x7f73b0165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b006fc70 0x7f73b0165c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 -- 172.21.15.22:0/3022589216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b00cf6c0 con 0x7f73b00ce260 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73add9b640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c450 tx=0x7f73a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4017070 con 0x7f73b00ce260 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a4005cf0 con 0x7f73b00ce260 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b0158d00 con 0x7f73b00ce260 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4007290 con 0x7f73b00ce260 2024-04-27T07:51:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b015a6e0 con 0x7f73b00ce260 2024-04-27T07:51:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b0159220 con 0x7f73b00ce260 2024-04-27T07:51:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73a401a070 con 0x7f73b00ce260 2024-04-27T07:51:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 0x7f739006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f73a4085d90 con 0x7f73b00ce260 2024-04-27T07:51:16.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.519+0000 7f73ae59c640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 0x7f739006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:16.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.523+0000 7f73ae59c640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 0x7f739006a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b930 tx=0x7f73a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:16.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.523+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73a4056350 con 0x7f73b00ce260 2024-04-27T07:51:16.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.723+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73b0068c30 con 0x7f73b00ce260 2024-04-27T07:51:16.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f739effd640 1 -- 172.21.15.22:0/3022589216 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f73a4055c50 con 0x7f73b00ce260 2024-04-27T07:51:16.731 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:51:16.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 msgr2=0x7f739006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 0x7f739006a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b930 tx=0x7f73a0008040 comp rx=0 tx=0).stop 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 msgr2=0x7f73b01583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c450 tx=0x7f73a400c910 comp rx=0 tx=0).stop 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 shutdown_connections 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7390068380 0x7f739006a840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73b0162da0 0x7f73b01fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73b00ce260 0x7f73b01583b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 --2- 172.21.15.22:0/3022589216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b006fc70 0x7f73b0165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:16.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 >> 172.21.15.22:0/3022589216 conn(0x7f73b0067f10 msgr2=0x7f73b0159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:16.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 shutdown_connections 2024-04-27T07:51:16.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:16.727+0000 7f73b4e90640 1 -- 172.21.15.22:0/3022589216 wait complete. 2024-04-27T07:51:16.750 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:51:16.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:51:16.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2024-04-27T07:51:16.750 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-27T07:51:16.751 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:16.751 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:51:16.751 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:51:16.751 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:51:16.751 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-27T07:51:16.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.751 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:16.758 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f0 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f1 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42514sec preferred_lft 42514sec 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::ec4:7aff:fe88:8085/64 scope link 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 02:42:99:0d:61:5b brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:16.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.761 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:51:16.812 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-04-27T07:51:16.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.814 DEBUG:teuthology.orchestra.run.smithi183:> route 2024-04-27T07:51:16.865 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2024-04-27T07:51:16.865 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:51:16.865 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-27T07:51:16.866 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:51:16.866 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:16.866 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:16.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-27T07:51:16.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-27T07:51:16.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-27T07:51:16.867 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:16.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:16.982 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:16.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.990 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2024-04-27T07:51:16.996 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-27T07:51:16.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:16.996 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2024-04-27T07:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-27T07:51:17.045 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:17.045 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:17.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:17.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-27T07:51:17.045 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:17.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:17.154 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:17.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:17.162 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-27T07:51:17.174 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 9a:e7:18:39:64:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::98e7:18ff:fe39:64d2/64 scope link 2024-04-27T07:51:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:17.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-27T07:51:17.234 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:51:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:51:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 9a:e7:18:39:64:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::98e7:18ff:fe39:64d2/64 scope link 2024-04-27T07:51:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:51:17.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-27T07:51:17.236 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:17.236 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:17.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-27T07:51:17.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-27T07:51:17.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-27T07:51:17.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-27T07:51:17.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-27T07:51:17.237 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:17.327 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:17.379 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:51:17.379 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2024-04-27T07:51:17.380 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.1 up 2024-04-27T07:51:17.380 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.1 master ceph-brx 2024-04-27T07:51:17.380 DEBUG:teuthology.orchestra.run.smithi183:> ') 2024-04-27T07:51:17.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:51:17.432 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:17.438 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-27T07:51:17.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:17.438 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:17.489 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-27T07:51:17.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:51:17.490 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:17.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:17.591 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:17.638 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:17.638 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:17.638 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:17.639 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:17.640 INFO:teuthology.orchestra.run.smithi183.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:17.641 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:17.641 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:17.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:17.642 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:17.689 INFO:teuthology.orchestra.run.smithi183.stdout:45 2024-04-27T07:51:17.689 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-27T07:51:17.690 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-27T07:51:17.734 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:17.749 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:17.797 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:17.797 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:17.797 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:17.797 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:17.797 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:17.798 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:17.799 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:17.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:17.802 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:17.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:2024-04-27T07:51:17.807+0000 7f2bdc980580 -1 init, newargv = 0x55fdaf664630 newargc=15 2024-04-27T07:51:17.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:ceph-fuse[33415]: starting ceph client 2024-04-27T07:51:17.816 INFO:teuthology.orchestra.run.smithi183.stdout:45 2024-04-27T07:51:17.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:ceph-fuse[33415]: starting fuse 2024-04-27T07:51:18.919 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2024-04-27T07:51:18.935 DEBUG:teuthology.orchestra.run.smithi183:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:51:18.982 INFO:teuthology.orchestra.run.smithi183.stdout:/sys 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/proc 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/dev 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/pts 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/run 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/ 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/security 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/shm 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/run/lock 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/cgroup 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/pstore 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/bpf 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/hugepages 2024-04-27T07:51:18.983 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/mqueue 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/debug 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/tracing 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/proc/fs/nfsd 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/fs/fuse/connections 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/sys/kernel/config 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/run/rpc_pipefs 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/run/user/1000 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-4 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-5 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-6 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/var/lib/ceph/osd/ceph-7 2024-04-27T07:51:18.984 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run.smithi183.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:18.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:18.985 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2024-04-27T07:51:19.033 INFO:teuthology.orchestra.run.smithi183.stdout:45 2024-04-27T07:51:19.034 INFO:teuthology.orchestra.run.smithi183.stdout:46 2024-04-27T07:51:19.034 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-27T07:51:19.034 DEBUG:teuthology.orchestra.run.smithi183:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:51:19.034 DEBUG:teuthology.orchestra.run.smithi183:> import glob 2024-04-27T07:51:19.034 DEBUG:teuthology.orchestra.run.smithi183:> import re 2024-04-27T07:51:19.034 DEBUG:teuthology.orchestra.run.smithi183:> import os 2024-04-27T07:51:19.034 DEBUG:teuthology.orchestra.run.smithi183:> import subprocess 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> def _find_admin_socket(client_name): 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> files = glob.glob(asok_path) 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> # Given a non-glob path, it better be there 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> if "*" not in asok_path: 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> assert(len(files) == 1) 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> return files[0] 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> for f in files: 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:51:19.035 DEBUG:teuthology.orchestra.run.smithi183:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> contents = proc_f.read() 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> if mountpoint in contents: 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> return f 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> print(_find_admin_socket("client.3")) 2024-04-27T07:51:19.036 DEBUG:teuthology.orchestra.run.smithi183:> ' 2024-04-27T07:51:19.156 INFO:teuthology.orchestra.run.smithi183.stdout:/var/run/ceph/ceph-client.3.33415.asok 2024-04-27T07:51:19.159 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-27T07:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:51:19.168 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33415.asok 2024-04-27T07:51:19.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33415.asok status 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "metadata": { 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "entity_id": "3", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "hostname": "smithi183", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "pid": "33415", 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "root": "/" 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_count": 0, 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_pinned_count": 0, 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "id": 4624, 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "inst": { 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "name": { 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "client", 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "num": 4624 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.2:0", 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 29341391 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2024-04-27T07:51:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.2:0", 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 29341391 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "inst_str": "client.4624 192.168.0.2:0/29341391", 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "addr_str": "192.168.0.2:0/29341391", 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "inode_count": 1, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_epoch": 8, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch": 21, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "blocklisted": false, 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "fs_name": "cephfs" 2024-04-27T07:51:19.326 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-27T07:51:19.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.338 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-04-27T07:51:19.338 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:19.345 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-04-27T07:51:19.345 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:19.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:19.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.403 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-04-27T07:51:19.403 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:19.453 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-04-27T07:51:19.453 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:19.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:19.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.514 DEBUG:teuthology.orchestra.run.smithi183:> stat --file-system '--printf=%T 2024-04-27T07:51:19.515 DEBUG:teuthology.orchestra.run.smithi183:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:19.522 INFO:teuthology.orchestra.run.smithi183.stdout:fuseblk 2024-04-27T07:51:19.522 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:19.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.523 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:19.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.578 DEBUG:teuthology.orchestra.run.smithi183:> stat --file-system '--printf=%T 2024-04-27T07:51:19.578 DEBUG:teuthology.orchestra.run.smithi183:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:19.626 INFO:teuthology.orchestra.run.smithi183.stdout:fuseblk 2024-04-27T07:51:19.626 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:19.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:19.627 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:19.682 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-27T07:51:19.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:19.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.827+0000 7f7adf1a2640 1 Processor -- start 2024-04-27T07:51:19.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.827+0000 7f7adf1a2640 1 -- start start 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adf1a2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ad815c850 0x7f7ad809b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adf1a2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad809aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adf1a2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad80d0670 con 0x7f7ad80d0190 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adf1a2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad80d0950 con 0x7f7ad815c850 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adf1a2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad80cd300 con 0x7f7ad809aa30 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ad815c850 0x7f7ad809b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7ad7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7ad80d0190 0x7f7ad80cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54884/0 (socket says 172.21.15.22:54884) 2024-04-27T07:51:19.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7ad7fff640 1 -- 172.21.15.22:0/258123011 learned_addr learned my addr 172.21.15.22:0/258123011 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:19.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 -- 172.21.15.22:0/258123011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 msgr2=0x7f7ad809aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:19.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/258123011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad809aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 -- 172.21.15.22:0/258123011 >> v1:172.21.15.22:6789/0 conn(0x7f7ad80d0190 legacy=0x7f7ad80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:19.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 -- 172.21.15.22:0/258123011 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad80cd5e0 con 0x7f7ad815c850 2024-04-27T07:51:19.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/258123011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815c850 0x7f7ad809b7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097c0 tx=0x7f7ac801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568114638ec02e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:19.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7ad77fe640 1 -- 172.21.15.22:0/258123011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b2070 con 0x7f7ad815c850 2024-04-27T07:51:19.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7ad77fe640 1 -- 172.21.15.22:0/258123011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac80ad640 con 0x7f7ad815c850 2024-04-27T07:51:19.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.831+0000 7f7ad77fe640 1 -- 172.21.15.22:0/258123011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80ada10 con 0x7f7ad815c850 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/258123011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815c850 msgr2=0x7f7ad809b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/258123011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815c850 0x7f7ad809b7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097c0 tx=0x7f7ac801c4a0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/258123011 shutdown_connections 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/258123011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad809aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/258123011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad815c850 0x7f7ad809b7e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/258123011 >> 172.21.15.22:0/258123011 conn(0x7f7ad80b40b0 msgr2=0x7f7ad80b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/258123011 shutdown_connections 2024-04-27T07:51:19.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/258123011 wait complete. 2024-04-27T07:51:19.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 Processor -- start 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- start start 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad8159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815c850 0x7f7ad80b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad809bea0 con 0x7f7ad80d0190 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad809bd20 con 0x7f7ad815c850 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad8169770 con 0x7f7ad809aa30 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46014/0 (socket says 172.21.15.22:46014) 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 -- 172.21.15.22:0/722743975 learned_addr learned my addr 172.21.15.22:0/722743975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad8159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:19.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 -- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 msgr2=0x7f7ad8159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad8159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7add73f640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815c850 0x7f7ad80b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 -- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815c850 msgr2=0x7f7ad80b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815c850 0x7f7ad80b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 -- 172.21.15.22:0/722743975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad8159f60 con 0x7f7ad80d0190 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad8159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad7fff640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c420 tx=0x7f7acc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc017070 con 0x7f7ad80d0190 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc005cf0 con 0x7f7ad80d0190 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad80b6210 con 0x7f7ad80d0190 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0072f0 con 0x7f7ad80d0190 2024-04-27T07:51:19.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.835+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad80b64b0 con 0x7f7ad80d0190 2024-04-27T07:51:19.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad8069340 con 0x7f7ad80d0190 2024-04-27T07:51:19.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7acc01a070 con 0x7f7ad80d0190 2024-04-27T07:51:19.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7ad57fa640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:19.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7acc0861c0 con 0x7f7ad80d0190 2024-04-27T07:51:19.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:19.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.839+0000 7f7adcf3e640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80af000 tx=0x7f7ac8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:19.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:19.843+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7acc021020 con 0x7f7ad80d0190 2024-04-27T07:51:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.007+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ad80cea60 con 0x7f7ad80d0190 2024-04-27T07:51:20.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.007+0000 7f7ad57fa640 1 -- 172.21.15.22:0/722743975 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f7acc0567f0 con 0x7f7ad80d0190 2024-04-27T07:51:20.014 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:20.014 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:20.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 msgr2=0x7f7ac406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80af000 tx=0x7f7ac8008040 comp rx=0 tx=0).stop 2024-04-27T07:51:20.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 msgr2=0x7f7ad8150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c420 tx=0x7f7acc00c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 shutdown_connections 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7ac4068380 0x7f7ac406a840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ad815c850 0x7f7ad80b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ad80d0190 0x7f7ad8150b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 --2- 172.21.15.22:0/722743975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ad809aa30 0x7f7ad8159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 >> 172.21.15.22:0/722743975 conn(0x7f7ad80b40b0 msgr2=0x7f7ad80cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 shutdown_connections 2024-04-27T07:51:20.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.011+0000 7f7adf1a2640 1 -- 172.21.15.22:0/722743975 wait complete. 2024-04-27T07:51:20.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:20.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:20.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 Processor -- start 2024-04-27T07:51:20.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 -- start start 2024-04-27T07:51:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f268c071f70 0x7f268c072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268c072890 con 0x7f268c06ee50 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268c0cdb10 con 0x7f268c0cffe0 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f2692c4f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f268c0ce0d0 con 0x7f268c071f70 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26911ec640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f268c06ee50 0x7f268c06f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54136/0 (socket says 172.21.15.22:54136) 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26911ec640 1 -- 172.21.15.22:0/2417060607 learned_addr learned my addr 172.21.15.22:0/2417060607 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 --2- 172.21.15.22:0/2417060607 >> v2:172.21.15.183:3300/0 conn(0x7f268c071f70 0x7f268c072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884684170 0 0) 0x7f268c072890 con 0x7f268c06ee50 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674003530 con 0x7f268c06ee50 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2845827508 0 0) 0x7f2674003530 con 0x7f268c06ee50 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 >> v2:172.21.15.183:3300/0 conn(0x7f268c071f70 msgr2=0x7f268c072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 --2- 172.21.15.22:0/2417060607 >> v2:172.21.15.183:3300/0 conn(0x7f268c071f70 0x7f268c072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 >> v1:172.21.15.183:6789/0 conn(0x7f268c0cffe0 legacy=0x7f268c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c0cddf0 con 0x7f268c06ee50 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f2684001680 con 0x7f268c06ee50 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 >> v1:172.21.15.22:6789/0 conn(0x7f268c06ee50 legacy=0x7f268c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 0x7f2674003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674004810 0x7f2674004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26740051d0 0x7f267400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f268c0ce0d0 con 0x7f26740039c0 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f267400c170 con 0x7f2674004810 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f267400c370 con 0x7f26740051d0 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 0x7f2674003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f268bfff640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674004810 0x7f2674004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26911ec640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26740051d0 0x7f267400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 -- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26740051d0 msgr2=0x7f267400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26740051d0 0x7f267400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 -- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674004810 msgr2=0x7f2674004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674004810 0x7f2674004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26909eb640 1 -- 172.21.15.22:0/2417060607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674003530 con 0x7f26740039c0 2024-04-27T07:51:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.159+0000 7f26911ec640 1 -- 172.21.15.22:0/2417060607 reap_dead start 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f26909eb640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 0x7f2674003de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fc0 tx=0x7f267c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efdc0942ee6be67 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0b2070 con 0x7f26740039c0 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267c0a5a70 con 0x7f26740039c0 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f268b7fe640 1 -- 172.21.15.22:0/2417060607 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c009c10 con 0x7f26740039c0 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 msgr2=0x7f2674003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 0x7f2674003de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fc0 tx=0x7f267c01cc80 comp rx=0 tx=0).stop 2024-04-27T07:51:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- 172.21.15.22:0/2417060607 shutdown_connections 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 --2- 172.21.15.22:0/2417060607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26740039c0 0x7f2674003de0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- 172.21.15.22:0/2417060607 >> 172.21.15.22:0/2417060607 conn(0x7f268c0b21d0 msgr2=0x7f268c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- 172.21.15.22:0/2417060607 shutdown_connections 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- 172.21.15.22:0/2417060607 wait complete. 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 Processor -- start 2024-04-27T07:51:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 -- start start 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.163+0000 7f2692c4f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26740039c0 0x7f268c1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f268c15bcb0 0x7f268c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f268c06dfd0 con 0x7f268c143070 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f268c0cd8c0 con 0x7f26740039c0 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f268c06fa40 con 0x7f268c15bcb0 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58268/0 (socket says 172.21.15.22:58268) 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 -- 172.21.15.22:0/3114584913 learned_addr learned my addr 172.21.15.22:0/3114584913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26909eb640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26740039c0 0x7f268c1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f268bfff640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f268c15bcb0 0x7f268c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 -- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f268c15bcb0 msgr2=0x7f268c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f268c15bcb0 0x7f268c15c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 -- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26740039c0 msgr2=0x7f268c1e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26740039c0 0x7f268c1e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 -- 172.21.15.22:0/3114584913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c143dc0 con 0x7f268c143070 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26911ec640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f268c06dec0 tx=0x7f268400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684007440 con 0x7f268c143070 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26840075e0 con 0x7f268c143070 2024-04-27T07:51:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1440a0 con 0x7f268c143070 2024-04-27T07:51:20.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684004a30 con 0x7f268c143070 2024-04-27T07:51:20.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c144660 con 0x7f268c143070 2024-04-27T07:51:20.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c144a40 con 0x7f268c143070 2024-04-27T07:51:20.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f268401b020 con 0x7f268c143070 2024-04-27T07:51:20.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 0x7f267806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2684086470 con 0x7f268c143070 2024-04-27T07:51:20.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.167+0000 7f26909eb640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 0x7f267806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.171+0000 7f26909eb640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 0x7f267806a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.171+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f268c144a40 con 0x7f268c143070 2024-04-27T07:51:20.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.335+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f268c04ff10 con 0x7f268c143070 2024-04-27T07:51:20.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.335+0000 7f26897fa640 1 -- 172.21.15.22:0/3114584913 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f2684056a30 con 0x7f268c143070 2024-04-27T07:51:20.342 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 msgr2=0x7f267806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 0x7f267806a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c004ac0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 msgr2=0x7f268c1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f268c06dec0 tx=0x7f268400d9b0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 shutdown_connections 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f26780683b0 0x7f267806a870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f268c143070 0x7f268c1434f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f268c15bcb0 0x7f268c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 --2- 172.21.15.22:0/3114584913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26740039c0 0x7f268c1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 >> 172.21.15.22:0/3114584913 conn(0x7f268c0b21d0 msgr2=0x7f268c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 shutdown_connections 2024-04-27T07:51:20.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.339+0000 7f2692c4f640 1 -- 172.21.15.22:0/3114584913 wait complete. 2024-04-27T07:51:20.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:20.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.475+0000 7fd37f91b640 1 Processor -- start 2024-04-27T07:51:20.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.475+0000 7fd37f91b640 1 -- start start 2024-04-27T07:51:20.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37f91b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd3780cee20 0x7fd3780cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37f91b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3780b5390 con 0x7fd37815ecd0 2024-04-27T07:51:20.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37f91b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3780b5670 con 0x7fd3780b4ee0 2024-04-27T07:51:20.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37f91b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd378073090 con 0x7fd3780cee20 2024-04-27T07:51:20.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37d6b7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd37815ecd0 0x7fd378072980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54142/0 (socket says 172.21.15.22:54142) 2024-04-27T07:51:20.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37d6b7640 1 -- 172.21.15.22:0/2121253260 learned_addr learned my addr 172.21.15.22:0/2121253260 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:20.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 --2- 172.21.15.22:0/2121253260 >> v2:172.21.15.183:3300/0 conn(0x7fd3780cee20 0x7fd3780cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929602902 0 0) 0x7fd3780b5390 con 0x7fd37815ecd0 2024-04-27T07:51:20.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd364003530 con 0x7fd37815ecd0 2024-04-27T07:51:20.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2987200472 0 0) 0x7fd364003530 con 0x7fd37815ecd0 2024-04-27T07:51:20.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 >> v2:172.21.15.183:3300/0 conn(0x7fd3780cee20 msgr2=0x7fd3780cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 --2- 172.21.15.22:0/2121253260 >> v2:172.21.15.183:3300/0 conn(0x7fd3780cee20 0x7fd3780cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 >> v1:172.21.15.183:6789/0 conn(0x7fd3780b4ee0 legacy=0x7fd3780b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3780ce340 con 0x7fd37815ecd0 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fd374001a00 con 0x7fd37815ecd0 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 >> v1:172.21.15.22:6789/0 conn(0x7fd37815ecd0 legacy=0x7fd378072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd364003a10 0x7fd364003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364004810 0x7fd364004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 0x7fd36400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd378073090 con 0x7fd3640051d0 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd36400c0d0 con 0x7fd364004810 2024-04-27T07:51:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd36400c2d0 con 0x7fd364003a10 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 0x7fd36400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37ceb6640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364004810 0x7fd364004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37d6b7640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd364003a10 0x7fd364003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 -- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd364003a10 msgr2=0x7fd364003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd364003a10 0x7fd364003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 -- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364004810 msgr2=0x7fd364004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364004810 0x7fd364004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 -- 172.21.15.22:0/2121253260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd364003530 con 0x7fd3640051d0 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 -- 172.21.15.22:0/2121253260 reap_dead start 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.479+0000 7fd37deb8640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 0x7fd36400b9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd370009ac0 tx=0x7fd37001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f1037a2a763057 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370009690 con 0x7fd3640051d0 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3700a5680 con 0x7fd3640051d0 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd36ffff640 1 -- 172.21.15.22:0/2121253260 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3700a5a50 con 0x7fd3640051d0 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 msgr2=0x7fd36400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 0x7fd36400b9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd370009ac0 tx=0x7fd37001ca90 comp rx=0 tx=0).stop 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- 172.21.15.22:0/2121253260 shutdown_connections 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 --2- 172.21.15.22:0/2121253260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3640051d0 0x7fd36400b9d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- 172.21.15.22:0/2121253260 >> 172.21.15.22:0/2121253260 conn(0x7fd3780b21d0 msgr2=0x7fd3780b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- 172.21.15.22:0/2121253260 shutdown_connections 2024-04-27T07:51:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- 172.21.15.22:0/2121253260 wait complete. 2024-04-27T07:51:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 Processor -- start 2024-04-27T07:51:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- start start 2024-04-27T07:51:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3640051d0 0x7fd378154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd378156630 0x7fd378145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780cf2b0 con 0x7fd37815ac00 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780ce900 con 0x7fd3640051d0 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37f91b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780cf430 con 0x7fd378156630 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37ceb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd378156630 0x7fd378145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58278/0 (socket says 172.21.15.22:58278) 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 -- 172.21.15.22:0/1005782440 learned_addr learned my addr 172.21.15.22:0/1005782440 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37d6b7640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3640051d0 0x7fd378154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 -- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd378156630 msgr2=0x7fd378145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd378156630 0x7fd378145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 -- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3640051d0 msgr2=0x7fd378154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3640051d0 0x7fd378154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.483+0000 7fd37deb8640 1 -- 172.21.15.22:0/1005782440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd378156350 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37deb8640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd37001c8c0 tx=0x7fd370019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3700bc070 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3700ad440 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd378155590 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370009690 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd378155b50 con 0x7fd37815ac00 2024-04-27T07:51:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd37804ff10 con 0x7fd37815ac00 2024-04-27T07:51:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd3700bf070 con 0x7fd37815ac00 2024-04-27T07:51:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 0x7fd35c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd370120330 con 0x7fd37815ac00 2024-04-27T07:51:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37d6b7640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 0x7fd35c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.487+0000 7fd37d6b7640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 0x7fd35c06a840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd374002240 tx=0x7fd374002190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.491+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd3700167a0 con 0x7fd37815ac00 2024-04-27T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.655+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3780cc320 con 0x7fd37815ac00 2024-04-27T07:51:20.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.655+0000 7fd36dffb640 1 -- 172.21.15.22:0/1005782440 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd3700f0170 con 0x7fd37815ac00 2024-04-27T07:51:20.661 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:20.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 msgr2=0x7fd35c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 0x7fd35c06a840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd374002240 tx=0x7fd374002190 comp rx=0 tx=0).stop 2024-04-27T07:51:20.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 msgr2=0x7fd37815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd37001c8c0 tx=0x7fd370019830 comp rx=0 tx=0).stop 2024-04-27T07:51:20.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 shutdown_connections 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd35c068380 0x7fd35c06a840 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37815ac00 0x7fd37815b0b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd378156630 0x7fd378145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 --2- 172.21.15.22:0/1005782440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3640051d0 0x7fd378154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 >> 172.21.15.22:0/1005782440 conn(0x7fd3780b21d0 msgr2=0x7fd37815f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 shutdown_connections 2024-04-27T07:51:20.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.659+0000 7fd37f91b640 1 -- 172.21.15.22:0/1005782440 wait complete. 2024-04-27T07:51:20.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:20.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 Processor -- start 2024-04-27T07:51:20.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 -- start start 2024-04-27T07:51:20.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e815eab0 0x7f77e80d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f77e80d4740 0x7f77e80d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77e80b6230 0x7f77e80b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f77e80d50e0 con 0x7f77e80b6230 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f77e80cf460 con 0x7f77e80d4740 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77ed19d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80cfdd0 con 0x7f77e815eab0 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77e80b6230 0x7f77e80b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77e80b6230 0x7f77e80b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58282/0 (socket says 172.21.15.22:58282) 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 -- 172.21.15.22:0/105166136 learned_addr learned my addr 172.21.15.22:0/105166136 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e659c640 1 --2- 172.21.15.22:0/105166136 >> v2:172.21.15.183:3300/0 conn(0x7f77e80d4740 0x7f77e80d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e6d9d640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e815eab0 0x7f77e80d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 -- 172.21.15.22:0/105166136 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e815eab0 msgr2=0x7f77e80d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e815eab0 0x7f77e80d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 -- 172.21.15.22:0/105166136 >> v2:172.21.15.183:3300/0 conn(0x7f77e80d4740 msgr2=0x7f77e80d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 --2- 172.21.15.22:0/105166136 >> v2:172.21.15.183:3300/0 conn(0x7f77e80d4740 0x7f77e80d4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.803+0000 7f77e759e640 1 -- 172.21.15.22:0/105166136 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e8163290 con 0x7f77e80b6230 2024-04-27T07:51:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77e759e640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e80b66c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77e806ddd0 tx=0x7f77e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670ac0e149d095fa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77e5d9b640 1 -- 172.21.15.22:0/105166136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e00ab620 con 0x7f77e80b6230 2024-04-27T07:51:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77e5d9b640 1 -- 172.21.15.22:0/105166136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e00b0070 con 0x7f77e80b6230 2024-04-27T07:51:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77e5d9b640 1 -- 172.21.15.22:0/105166136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e00ab9e0 con 0x7f77e80b6230 2024-04-27T07:51:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- 172.21.15.22:0/105166136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 msgr2=0x7f77e80b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e80b66c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77e806ddd0 tx=0x7f77e001c460 comp rx=0 tx=0).stop 2024-04-27T07:51:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- 172.21.15.22:0/105166136 shutdown_connections 2024-04-27T07:51:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e80b66c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 --2- 172.21.15.22:0/105166136 >> v2:172.21.15.183:3300/0 conn(0x7f77e80d4740 0x7f77e80d4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 --2- 172.21.15.22:0/105166136 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e815eab0 0x7f77e80d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- 172.21.15.22:0/105166136 >> 172.21.15.22:0/105166136 conn(0x7f77e8067d10 msgr2=0x7f77e8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- 172.21.15.22:0/105166136 shutdown_connections 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- 172.21.15.22:0/105166136 wait complete. 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 Processor -- start 2024-04-27T07:51:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.807+0000 7f77ed19d640 1 -- start start 2024-04-27T07:51:20.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77e80d4740 0x7f77e8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e81561f0 0x7f77e814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80b3ae0 con 0x7f77e80b6230 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80cf8b0 con 0x7f77e80d4740 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80b3c60 con 0x7f77e81561f0 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58288/0 (socket says 172.21.15.22:58288) 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 -- 172.21.15.22:0/1165342210 learned_addr learned my addr 172.21.15.22:0/1165342210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e659c640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77e80d4740 0x7f77e8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e759e640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e81561f0 0x7f77e814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 -- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e81561f0 msgr2=0x7f77e814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e81561f0 0x7f77e814d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 -- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77e80d4740 msgr2=0x7f77e8155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77e80d4740 0x7f77e8155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 -- 172.21.15.22:0/1165342210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81567b0 con 0x7f77e80b6230 2024-04-27T07:51:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e6d9d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009db0 tx=0x7f77d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8018020 con 0x7f77e80b6230 2024-04-27T07:51:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8014480 con 0x7f77e80b6230 2024-04-27T07:51:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8014850 con 0x7f77e80b6230 2024-04-27T07:51:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e8156a90 con 0x7f77e80b6230 2024-04-27T07:51:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e8157050 con 0x7f77e80b6230 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e804ff10 con 0x7f77e80b6230 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f77d801b020 con 0x7f77e80b6230 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 0x7f77c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77d80874f0 con 0x7f77e80b6230 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.811+0000 7f77e659c640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 0x7f77c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:20.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.815+0000 7f77e659c640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 0x7f77c806a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77e8150a40 tx=0x7f77d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:20.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.819+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77d8022070 con 0x7f77e80b6230 2024-04-27T07:51:20.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.979+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77e80b49e0 con 0x7f77e80b6230 2024-04-27T07:51:20.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.979+0000 7f77df7fe640 1 -- 172.21.15.22:0/1165342210 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f77d8010040 con 0x7f77e80b6230 2024-04-27T07:51:20.985 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 msgr2=0x7f77c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 0x7f77c806a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77e8150a40 tx=0x7f77d0009190 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 msgr2=0x7f77e814ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009db0 tx=0x7f77d800c490 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 shutdown_connections 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f77c8068380 0x7f77c806a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77e81561f0 0x7f77e814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77e80d4740 0x7f77e8155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 --2- 172.21.15.22:0/1165342210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e80b6230 0x7f77e814ccc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 >> 172.21.15.22:0/1165342210 conn(0x7f77e8067d10 msgr2=0x7f77e8151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:20.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 shutdown_connections 2024-04-27T07:51:20.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:20.983+0000 7f77ed19d640 1 -- 172.21.15.22:0/1165342210 wait complete. 2024-04-27T07:51:21.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:21.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 Processor -- start 2024-04-27T07:51:21.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 -- start start 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa8e4156b40 0x7fa8e415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e40b5cc0 0x7fa8e40b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa8e40b5060 0x7fa8e406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e415dbf0 con 0x7fa8e4156b40 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e40aff80 con 0x7fa8e40b5060 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8eb328640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e40b0220 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa8e4156b40 0x7fa8e415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa8e4156b40 0x7fa8e415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58296/0 (socket says 172.21.15.22:58296) 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 -- 172.21.15.22:0/871423697 learned_addr learned my addr 172.21.15.22:0/871423697 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 -- 172.21.15.22:0/871423697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e40b5cc0 msgr2=0x7fa8e40b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/871423697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e40b5cc0 0x7fa8e40b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 -- 172.21.15.22:0/871423697 >> v2:172.21.15.183:3300/0 conn(0x7fa8e40b5060 msgr2=0x7fa8e406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/871423697 >> v2:172.21.15.183:3300/0 conn(0x7fa8e40b5060 0x7fa8e406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.119+0000 7fa8e90c4640 1 -- 172.21.15.22:0/871423697 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e40b4040 con 0x7fa8e4156b40 2024-04-27T07:51:21.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/871423697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e4156b40 0x7fa8e415d6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40d1da0 tx=0x7fa8dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb44acb94640adff server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e3fff640 1 -- 172.21.15.22:0/871423697 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0ab640 con 0x7fa8e4156b40 2024-04-27T07:51:21.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e3fff640 1 -- 172.21.15.22:0/871423697 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc0b0070 con 0x7fa8e4156b40 2024-04-27T07:51:21.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e3fff640 1 -- 172.21.15.22:0/871423697 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0aba00 con 0x7fa8e4156b40 2024-04-27T07:51:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- 172.21.15.22:0/871423697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e4156b40 msgr2=0x7fa8e415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- 172.21.15.22:0/871423697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e4156b40 0x7fa8e415d6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40d1da0 tx=0x7fa8dc01c460 comp rx=0 tx=0).stop 2024-04-27T07:51:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- 172.21.15.22:0/871423697 shutdown_connections 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- 172.21.15.22:0/871423697 >> v2:172.21.15.183:3300/0 conn(0x7fa8e40b5060 0x7fa8e406bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- 172.21.15.22:0/871423697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e40b5cc0 0x7fa8e40b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- 172.21.15.22:0/871423697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e4156b40 0x7fa8e415d6b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- 172.21.15.22:0/871423697 >> 172.21.15.22:0/871423697 conn(0x7fa8e40d0cc0 msgr2=0x7fa8e40d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- 172.21.15.22:0/871423697 shutdown_connections 2024-04-27T07:51:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- 172.21.15.22:0/871423697 wait complete. 2024-04-27T07:51:21.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 Processor -- start 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- start start 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 0x7fa8e409e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 0x7fa8e415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e41570a0 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4156f20 con 0x7fa8e40b5060 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8eb328640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e4160220 con 0x7fa8e4156b40 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58298/0 (socket says 172.21.15.22:58298) 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 -- 172.21.15.22:0/1255132664 learned_addr learned my addr 172.21.15.22:0/1255132664 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 0x7fa8e409e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e98c5640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 0x7fa8e415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 -- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 msgr2=0x7fa8e415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 0x7fa8e415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 -- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 msgr2=0x7fa8e409e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 0x7fa8e409e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e88c3640 1 -- 172.21.15.22:0/1255132664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e409c1a0 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 0x7fa8e409e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.123+0000 7fa8e98c5640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 0x7fa8e415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e88c3640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b480 tx=0x7fa8d800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8014070 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d8010610 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e409dbc0 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8010a00 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e409e180 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e409d9b0 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa8d8017070 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 0x7fa8d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa8d8086730 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 0x7fa8d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.127+0000 7fa8e90c4640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 0x7fa8d006a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0af000 tx=0x7fa8dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.131+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa8d8056e60 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.295+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8e4069340 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.295+0000 7fa8e1ffb640 1 -- 172.21.15.22:0/1255132664 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fa8d8084e20 con 0x7fa8e40b5cc0 2024-04-27T07:51:21.303 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:21.303 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 msgr2=0x7fa8d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 0x7fa8d006a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0af000 tx=0x7fa8dc008040 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 msgr2=0x7fa8e4150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b480 tx=0x7fa8d800b940 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 shutdown_connections 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa8d0068380 0x7fa8d006a840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8e4156b40 0x7fa8e415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8e40b5cc0 0x7fa8e4150950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 --2- 172.21.15.22:0/1255132664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8e40b5060 0x7fa8e409e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 >> 172.21.15.22:0/1255132664 conn(0x7fa8e40d0cc0 msgr2=0x7fa8e4155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 shutdown_connections 2024-04-27T07:51:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.299+0000 7fa8eb328640 1 -- 172.21.15.22:0/1255132664 wait complete. 2024-04-27T07:51:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:21.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:21.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 Processor -- start 2024-04-27T07:51:21.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 -- start start 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42bc0b3a10 0x7f42bc0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42bc0b5390 con 0x7f42bc15ecd0 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f42bc073090 con 0x7f42bc0b3a10 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c2679640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc0b5670 con 0x7f42bc0b4ee0 2024-04-27T07:51:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42bbfff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f42bc15ecd0 0x7f42bc072980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:40550/0 (socket says 172.21.15.22:40550) 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42bbfff640 1 -- 172.21.15.22:0/1329854917 learned_addr learned my addr 172.21.15.22:0/1329854917 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c0c16640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42bc0b3a10 0x7f42bc0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c0c16640 1 -- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 msgr2=0x7f42bc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c0c16640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c0c16640 1 -- 172.21.15.22:0/1329854917 >> v1:172.21.15.183:6789/0 conn(0x7f42bc15ecd0 legacy=0x7f42bc072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42c0c16640 1 -- 172.21.15.22:0/1329854917 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc0ce340 con 0x7f42bc0b3a10 2024-04-27T07:51:21.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.439+0000 7f42b3fff640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:21.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c0c16640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc0cee20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009dc0 tx=0x7f42b401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4580655acbd20a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42bb7fe640 1 -- 172.21.15.22:0/1329854917 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40b2070 con 0x7f42bc0b3a10 2024-04-27T07:51:21.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42bb7fe640 1 -- 172.21.15.22:0/1329854917 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b40a58f0 con 0x7f42bc0b3a10 2024-04-27T07:51:21.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42bb7fe640 1 -- 172.21.15.22:0/1329854917 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40a5cc0 con 0x7f42bc0b3a10 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 -- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 msgr2=0x7f42bc0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc0cee20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009dc0 tx=0x7f42b401c920 comp rx=0 tx=0).stop 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 -- 172.21.15.22:0/1329854917 shutdown_connections 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc0cee20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 --2- 172.21.15.22:0/1329854917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 -- 172.21.15.22:0/1329854917 >> 172.21.15.22:0/1329854917 conn(0x7f42bc0b21d0 msgr2=0x7f42bc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 -- 172.21.15.22:0/1329854917 shutdown_connections 2024-04-27T07:51:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 -- 172.21.15.22:0/1329854917 wait complete. 2024-04-27T07:51:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.443+0000 7f42c2679640 1 Processor -- start 2024-04-27T07:51:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- start start 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1688d0 con 0x7f42bc15ecd0 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc168750 con 0x7f42bc0b3a10 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc168a50 con 0x7f42bc0b4ee0 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58304/0 (socket says 172.21.15.22:58304) 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 -- 172.21.15.22:0/3122354595 learned_addr learned my addr 172.21.15.22:0/3122354595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42b3fff640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42bbfff640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 -- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 msgr2=0x7f42bc153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 -- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 msgr2=0x7f42bc150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 -- 172.21.15.22:0/3122354595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1507a0 con 0x7f42bc15ecd0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c0c16640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b0040 tx=0x7f42b40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40b2050 con 0x7f42bc15ecd0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b40be070 con 0x7f42bc15ecd0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40ada70 con 0x7f42bc15ecd0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc14f340 con 0x7f42bc15ecd0 2024-04-27T07:51:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc14f900 con 0x7f42bc15ecd0 2024-04-27T07:51:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.447+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42bc04ff10 con 0x7f42bc15ecd0 2024-04-27T07:51:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.451+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f42b40a57b0 con 0x7f42bc15ecd0 2024-04-27T07:51:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.451+0000 7f42b97fa640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 0x7f429c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.451+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f42b4121a40 con 0x7f42bc15ecd0 2024-04-27T07:51:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.451+0000 7f42bbfff640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 0x7f429c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.451+0000 7f42bbfff640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 0x7f429c06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42bc15b0d0 tx=0x7f42ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.455+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f42b4016540 con 0x7f42bc15ecd0 2024-04-27T07:51:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.607+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42bc0cfc60 con 0x7f42bc15ecd0 2024-04-27T07:51:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.607+0000 7f42b97fa640 1 -- 172.21.15.22:0/3122354595 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f42b40ab340 con 0x7f42bc15ecd0 2024-04-27T07:51:21.615 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 msgr2=0x7f429c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 0x7f429c06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42bc15b0d0 tx=0x7f42ac009040 comp rx=0 tx=0).stop 2024-04-27T07:51:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 msgr2=0x7f42bc0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b0040 tx=0x7f42b40196c0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 shutdown_connections 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f429c068380 0x7f429c06a840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42bc15ecd0 0x7f42bc0b0d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b4ee0 0x7f42bc153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 --2- 172.21.15.22:0/3122354595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc0b3a10 0x7f42bc150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 >> 172.21.15.22:0/3122354595 conn(0x7f42bc0b21d0 msgr2=0x7f42bc158590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 shutdown_connections 2024-04-27T07:51:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.611+0000 7f42c2679640 1 -- 172.21.15.22:0/3122354595 wait complete. 2024-04-27T07:51:21.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:21.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 Processor -- start 2024-04-27T07:51:21.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 -- start start 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c8415c740 0x7f9c84160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 0x7f9c840af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9c840af5d0 0x7f9c8406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9c84161240 con 0x7f9c840af5d0 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8406ead0 con 0x7f9c840ce1d0 2024-04-27T07:51:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8adf2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8406edb0 con 0x7f9c8415c740 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9c840af5d0 0x7f9c8406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9c840af5d0 0x7f9c8406fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58306/0 (socket says 172.21.15.22:58306) 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 -- 172.21.15.22:0/556775752 learned_addr learned my addr 172.21.15.22:0/556775752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c83fff640 1 --2- 172.21.15.22:0/556775752 >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 0x7f9c840af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c88b8e640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c8415c740 0x7f9c84160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 -- 172.21.15.22:0/556775752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c8415c740 msgr2=0x7f9c84160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c8415c740 0x7f9c84160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 -- 172.21.15.22:0/556775752 >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 msgr2=0x7f9c840af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 --2- 172.21.15.22:0/556775752 >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 0x7f9c840af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c8938f640 1 -- 172.21.15.22:0/556775752 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8406f090 con 0x7f9c840af5d0 2024-04-27T07:51:21.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.755+0000 7f9c83fff640 1 --2- 172.21.15.22:0/556775752 >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 0x7f9c840af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:21.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8938f640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c8406fb70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c840b70c0 tx=0x7f9c7c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df2ad8bf6f6877e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c837fe640 1 -- 172.21.15.22:0/556775752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0b5070 con 0x7f9c840af5d0 2024-04-27T07:51:21.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c837fe640 1 -- 172.21.15.22:0/556775752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c0a89d0 con 0x7f9c840af5d0 2024-04-27T07:51:21.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c837fe640 1 -- 172.21.15.22:0/556775752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0a8da0 con 0x7f9c840af5d0 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- 172.21.15.22:0/556775752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 msgr2=0x7f9c8406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c8406fb70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c840b70c0 tx=0x7f9c7c01ed10 comp rx=0 tx=0).stop 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- 172.21.15.22:0/556775752 shutdown_connections 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c8406fb70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/556775752 >> v2:172.21.15.183:3300/0 conn(0x7f9c840ce1d0 0x7f9c840af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/556775752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c8415c740 0x7f9c84160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- 172.21.15.22:0/556775752 >> 172.21.15.22:0/556775752 conn(0x7f9c84073700 msgr2=0x7f9c8406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- 172.21.15.22:0/556775752 shutdown_connections 2024-04-27T07:51:21.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- 172.21.15.22:0/556775752 wait complete. 2024-04-27T07:51:21.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 Processor -- start 2024-04-27T07:51:21.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- start start 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 0x7f9c8415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8415c740 0x7f9c8413fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415cb20 con 0x7f9c840af5d0 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8406f650 con 0x7f9c8415c740 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c8adf2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840cd4c0 con 0x7f9c840ce1d0 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c88b8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c88b8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58318/0 (socket says 172.21.15.22:58318) 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.759+0000 7f9c88b8e640 1 -- 172.21.15.22:0/4132378517 learned_addr learned my addr 172.21.15.22:0/4132378517 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c8938f640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8415c740 0x7f9c8413fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c83fff640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 0x7f9c8415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 -- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 msgr2=0x7f9c8415e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 0x7f9c8415e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 -- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8415c740 msgr2=0x7f9c8413fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8415c740 0x7f9c8413fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 -- 172.21.15.22:0/4132378517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8415e9e0 con 0x7f9c840af5d0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c83fff640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 0x7f9c8415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c88b8e640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400ccd0 tx=0x7f9c7400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c740039a0 con 0x7f9c840af5d0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c74003b40 con 0x7f9c840af5d0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8415ec80 con 0x7f9c840af5d0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74004020 con 0x7f9c840af5d0 2024-04-27T07:51:21.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8415f240 con 0x7f9c840af5d0 2024-04-27T07:51:21.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8404ff10 con 0x7f9c840af5d0 2024-04-27T07:51:21.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9c740041c0 con 0x7f9c840af5d0 2024-04-27T07:51:21.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 0x7f9c7006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:21.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c74086dd0 con 0x7f9c840af5d0 2024-04-27T07:51:21.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c83fff640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 0x7f9c7006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:21.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.763+0000 7f9c83fff640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 0x7f9c7006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c8414e730 tx=0x7f9c78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:21.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.767+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c74014030 con 0x7f9c840af5d0 2024-04-27T07:51:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c8415f940 con 0x7f9c840af5d0 2024-04-27T07:51:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c817fa640 1 -- 172.21.15.22:0/4132378517 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f9c74088020 con 0x7f9c840af5d0 2024-04-27T07:51:21.936 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:21.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 msgr2=0x7f9c7006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 0x7f9c7006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c8414e730 tx=0x7f9c78008040 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 msgr2=0x7f9c841432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400ccd0 tx=0x7f9c7400a5c0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 shutdown_connections 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9c70068380 0x7f9c7006a840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8415c740 0x7f9c8413fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9c840ce1d0 0x7f9c8415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 --2- 172.21.15.22:0/4132378517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c840af5d0 0x7f9c841432d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 >> 172.21.15.22:0/4132378517 conn(0x7f9c84073700 msgr2=0x7f9c840ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 shutdown_connections 2024-04-27T07:51:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:21.931+0000 7f9c8adf2640 1 -- 172.21.15.22:0/4132378517 wait complete. 2024-04-27T07:51:21.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:22.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 Processor -- start 2024-04-27T07:51:22.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 -- start start 2024-04-27T07:51:22.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff5f815d520 0x7ff5f8163e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80cd0f0 0x7ff5f80b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff5f80b5ac0 0x7ff5f81669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f815d900 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f8164360 con 0x7ff5f815d520 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5fd665640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f80af250 con 0x7ff5f80cd0f0 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff5f80b5ac0 0x7ff5f81669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff5f80b5ac0 0x7ff5f81669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58334/0 (socket says 172.21.15.22:58334) 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 -- 172.21.15.22:0/978152974 learned_addr learned my addr 172.21.15.22:0/978152974 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f67fc640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80cd0f0 0x7ff5f80b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f6ffd640 1 --2- 172.21.15.22:0/978152974 >> v2:172.21.15.183:3300/0 conn(0x7ff5f815d520 0x7ff5f8163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 -- 172.21.15.22:0/978152974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80cd0f0 msgr2=0x7ff5f80b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80cd0f0 0x7ff5f80b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 -- 172.21.15.22:0/978152974 >> v2:172.21.15.183:3300/0 conn(0x7ff5f815d520 msgr2=0x7ff5f8163e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/978152974 >> v2:172.21.15.183:3300/0 conn(0x7ff5f815d520 0x7ff5f8163e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.075+0000 7ff5f77fe640 1 -- 172.21.15.22:0/978152974 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f80af530 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f80b5ac0 0x7ff5f81669e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80b3b90 tx=0x7ff5e801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d845d63240f0452 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f5ffb640 1 -- 172.21.15.22:0/978152974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80b5070 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f5ffb640 1 -- 172.21.15.22:0/978152974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e80a89d0 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f5ffb640 1 -- 172.21.15.22:0/978152974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80a8da0 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- 172.21.15.22:0/978152974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f80b5ac0 msgr2=0x7ff5f81669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f80b5ac0 0x7ff5f81669e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80b3b90 tx=0x7ff5e801ed10 comp rx=0 tx=0).stop 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- 172.21.15.22:0/978152974 shutdown_connections 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f80b5ac0 0x7ff5f81669e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- 172.21.15.22:0/978152974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80cd0f0 0x7ff5f80b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- 172.21.15.22:0/978152974 >> v2:172.21.15.183:3300/0 conn(0x7ff5f815d520 0x7ff5f8163e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- 172.21.15.22:0/978152974 >> 172.21.15.22:0/978152974 conn(0x7ff5f8073700 msgr2=0x7ff5f806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- 172.21.15.22:0/978152974 shutdown_connections 2024-04-27T07:51:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- 172.21.15.22:0/978152974 wait complete. 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 Processor -- start 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- start start 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 0x7ff5f8154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 0x7ff5f80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80afdd0 con 0x7ff5f815d520 2024-04-27T07:51:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80afaf0 con 0x7ff5f80cd0f0 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5fd665640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f80b02f0 con 0x7ff5f80b5ac0 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58344/0 (socket says 172.21.15.22:58344) 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.079+0000 7ff5f77fe640 1 -- 172.21.15.22:0/1211372594 learned_addr learned my addr 172.21.15.22:0/1211372594 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f67fc640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 0x7ff5f80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f6ffd640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 0x7ff5f8154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 -- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 msgr2=0x7ff5f8154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 0x7ff5f8154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 -- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 msgr2=0x7ff5f80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 0x7ff5f80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 -- 172.21.15.22:0/1211372594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8069790 con 0x7ff5f815d520 2024-04-27T07:51:22.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f67fc640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 0x7ff5f80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f6ffd640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 0x7ff5f8154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b3040 tx=0x7ff5e801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80c0070 con 0x7ff5f815d520 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e80b09f0 con 0x7ff5f815d520 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f80755a0 con 0x7ff5f815d520 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80b5070 con 0x7ff5f815d520 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f815dbb0 con 0x7ff5f815d520 2024-04-27T07:51:22.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f804ff10 con 0x7ff5f815d520 2024-04-27T07:51:22.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff5e80c3070 con 0x7ff5f815d520 2024-04-27T07:51:22.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 0x7ff5d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e8123d10 con 0x7ff5f815d520 2024-04-27T07:51:22.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f6ffd640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 0x7ff5d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.083+0000 7ff5f6ffd640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 0x7ff5d806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80cd080 tx=0x7ff5f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.087+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5e8018870 con 0x7ff5f815d520 2024-04-27T07:51:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5f80cfcc0 con 0x7ff5f815d520 2024-04-27T07:51:22.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5ef7fe640 1 -- 172.21.15.22:0/1211372594 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff5e80f42d0 con 0x7ff5f815d520 2024-04-27T07:51:22.253 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 msgr2=0x7ff5d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 0x7ff5d806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80cd080 tx=0x7ff5f0008040 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 msgr2=0x7ff5f80ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b3040 tx=0x7ff5e801bab0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 shutdown_connections 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff5d8068380 0x7ff5d806a840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5f815d520 0x7ff5f80ced70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f80cd0f0 0x7ff5f80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 --2- 172.21.15.22:0/1211372594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5f80b5ac0 0x7ff5f8154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.247+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 >> 172.21.15.22:0/1211372594 conn(0x7ff5f8073700 msgr2=0x7ff5f8157b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.251+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 shutdown_connections 2024-04-27T07:51:22.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.251+0000 7ff5fd665640 1 -- 172.21.15.22:0/1211372594 wait complete. 2024-04-27T07:51:22.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 Processor -- start 2024-04-27T07:51:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 -- start start 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 0x7f65a806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 0x7f65a80cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f65a80cfc40 0x7f65a815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f65a806fb00 con 0x7f65a80cfc40 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80720d0 con 0x7f65a8144dd0 2024-04-27T07:51:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65af134640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f65a8072370 con 0x7f65a806f6a0 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f65a80cfc40 0x7f65a815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f65a80cfc40 0x7f65a815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58352/0 (socket says 172.21.15.22:58352) 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 -- 172.21.15.22:0/193970456 learned_addr learned my addr 172.21.15.22:0/193970456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65a7fff640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 0x7f65a80cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65aced0640 1 --2- 172.21.15.22:0/193970456 >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 0x7f65a806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 -- 172.21.15.22:0/193970456 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 msgr2=0x7f65a80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 0x7f65a80cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 -- 172.21.15.22:0/193970456 >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 msgr2=0x7f65a806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 --2- 172.21.15.22:0/193970456 >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 0x7f65a806f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65ad6d1640 1 -- 172.21.15.22:0/193970456 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a8072650 con 0x7f65a80cfc40 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65aced0640 1 --2- 172.21.15.22:0/193970456 >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 0x7f65a806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.387+0000 7f65a7fff640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 0x7f65a80cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65ad6d1640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815baf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65a80b3da0 tx=0x7f65a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5e14f1a3ed0260 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a77fe640 1 -- 172.21.15.22:0/193970456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00b1070 con 0x7f65a80cfc40 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a77fe640 1 -- 172.21.15.22:0/193970456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a00ac630 con 0x7f65a80cfc40 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a77fe640 1 -- 172.21.15.22:0/193970456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00a8050 con 0x7f65a80cfc40 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/193970456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 msgr2=0x7f65a815baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815baf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65a80b3da0 tx=0x7f65a001c520 comp rx=0 tx=0).stop 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/193970456 shutdown_connections 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815baf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- 172.21.15.22:0/193970456 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a806f6a0 0x7f65a80cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- 172.21.15.22:0/193970456 >> v2:172.21.15.183:3300/0 conn(0x7f65a8144dd0 0x7f65a806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/193970456 >> 172.21.15.22:0/193970456 conn(0x7f65a8067f10 msgr2=0x7f65a806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/193970456 shutdown_connections 2024-04-27T07:51:22.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/193970456 wait complete. 2024-04-27T07:51:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 Processor -- start 2024-04-27T07:51:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- start start 2024-04-27T07:51:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65a806f6a0 0x7f65a8141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a8144dd0 0x7f65a813e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80b3dd0 con 0x7f65a80cfc40 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80d00d0 con 0x7f65a806f6a0 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f65a80b3f50 con 0x7f65a8144dd0 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58354/0 (socket says 172.21.15.22:58354) 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 -- 172.21.15.22:0/2842673518 learned_addr learned my addr 172.21.15.22:0/2842673518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 -- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a8144dd0 msgr2=0x7f65a813e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a8144dd0 0x7f65a813e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65aced0640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65a806f6a0 0x7f65a8141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 -- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65a806f6a0 msgr2=0x7f65a8141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65a806f6a0 0x7f65a8141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 -- 172.21.15.22:0/2842673518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a8140a10 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a7fff640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f659800c7c0 tx=0x7f659800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598007a30 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598007bd0 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a8141ba0 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.391+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65980113f0 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a813fc50 con 0x7f65a80cfc40 2024-04-27T07:51:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a8068c30 con 0x7f65a80cfc40 2024-04-27T07:51:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6598011590 con 0x7f65a80cfc40 2024-04-27T07:51:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65a57fa640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 0x7f658806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6598086b60 con 0x7f65a80cfc40 2024-04-27T07:51:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65aced0640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 0x7f658806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.395+0000 7f65aced0640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 0x7f658806a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65a81f8560 tx=0x7f65900083d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.399+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6598015020 con 0x7f65a80cfc40 2024-04-27T07:51:22.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.559+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65a8142000 con 0x7f65a80cfc40 2024-04-27T07:51:22.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65a57fa640 1 -- 172.21.15.22:0/2842673518 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f65980571b0 con 0x7f65a80cfc40 2024-04-27T07:51:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:22.568 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 msgr2=0x7f658806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 0x7f658806a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65a81f8560 tx=0x7f65900083d0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 msgr2=0x7f65a815a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f659800c7c0 tx=0x7f659800cc80 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 shutdown_connections 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6588068380 0x7f658806a840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65a8144dd0 0x7f65a813e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a80cfc40 0x7f65a815a980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 --2- 172.21.15.22:0/2842673518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65a806f6a0 0x7f65a8141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.563+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 >> 172.21.15.22:0/2842673518 conn(0x7f65a8067f10 msgr2=0x7f65a806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.567+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 shutdown_connections 2024-04-27T07:51:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.567+0000 7f65af134640 1 -- 172.21.15.22:0/2842673518 wait complete. 2024-04-27T07:51:22.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:22.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:22.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.703+0000 7f70c7ef8640 1 Processor -- start 2024-04-27T07:51:22.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.703+0000 7f70c7ef8640 1 -- start start 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c7ef8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70c00afd00 0x7f70c00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c7ef8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 0x7f70c006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c7ef8640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c006e070 con 0x7f70c0161f30 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c7ef8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f70c0072400 con 0x7f70c00afd00 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c7ef8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f70c00726c0 con 0x7f70c0071f70 2024-04-27T07:51:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70c00afd00 0x7f70c00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70c00afd00 0x7f70c00b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58368/0 (socket says 172.21.15.22:58368) 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 -- 172.21.15.22:0/3051407576 learned_addr learned my addr 172.21.15.22:0/3051407576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c6495640 1 --2- 172.21.15.22:0/3051407576 >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 0x7f70c006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 -- 172.21.15.22:0/3051407576 >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 msgr2=0x7f70c006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 --2- 172.21.15.22:0/3051407576 >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 0x7f70c006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 -- 172.21.15.22:0/3051407576 >> v1:172.21.15.183:6789/0 conn(0x7f70c0161f30 legacy=0x7f70c00af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 -- 172.21.15.22:0/3051407576 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c00729a0 con 0x7f70c00afd00 2024-04-27T07:51:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c6495640 1 --2- 172.21.15.22:0/3051407576 >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 0x7f70c006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:22.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c5493640 1 --2- 172.21.15.22:0/3051407576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c00b34d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009870 tx=0x7f70b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45229eceb37cfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c4c92640 1 -- 172.21.15.22:0/3051407576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2070 con 0x7f70c00afd00 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c4c92640 1 -- 172.21.15.22:0/3051407576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b00a9070 con 0x7f70c00afd00 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.707+0000 7f70c4c92640 1 -- 172.21.15.22:0/3051407576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00ad880 con 0x7f70c00afd00 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3051407576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 msgr2=0x7f70c00b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3051407576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c00b34d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009870 tx=0x7f70b001c450 comp rx=0 tx=0).stop 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3051407576 shutdown_connections 2024-04-27T07:51:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3051407576 >> v2:172.21.15.183:3300/0 conn(0x7f70c0071f70 0x7f70c006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3051407576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c00b34d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3051407576 >> 172.21.15.22:0/3051407576 conn(0x7f70c00b21d0 msgr2=0x7f70c00b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3051407576 shutdown_connections 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3051407576 wait complete. 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 Processor -- start 2024-04-27T07:51:22.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- start start 2024-04-27T07:51:22.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70c0071f70 0x7f70c0157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c0161f30 0x7f70c00b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70c006f6d0 con 0x7f70c00afd00 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70c006f550 con 0x7f70c0161f30 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c7ef8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f70c006f850 con 0x7f70c0071f70 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58382/0 (socket says 172.21.15.22:58382) 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 -- 172.21.15.22:0/3486749346 learned_addr learned my addr 172.21.15.22:0/3486749346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c6495640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c0161f30 0x7f70c00b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5c94640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70c0071f70 0x7f70c0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 -- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70c0071f70 msgr2=0x7f70c0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70c0071f70 0x7f70c0157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 -- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c0161f30 msgr2=0x7f70c00b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c0161f30 0x7f70c00b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 -- 172.21.15.22:0/3486749346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c0157a30 con 0x7f70c00afd00 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70c5493640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b0040 tx=0x7f70b001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2040 con 0x7f70c00afd00 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b00be070 con 0x7f70c00afd00 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.711+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b8660 con 0x7f70c00afd00 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c0157d10 con 0x7f70c00afd00 2024-04-27T07:51:22.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01582d0 con 0x7f70c00afd00 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c004ff10 con 0x7f70c00afd00 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70b00ab070 con 0x7f70c00afd00 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70b67fc640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f70b0120b50 con 0x7f70c00afd00 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70c5c94640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.715+0000 7f70c5c94640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 0x7f70a406a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f70c0070450 tx=0x7f70bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:22.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.719+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70b00160d0 con 0x7f70c00afd00 2024-04-27T07:51:22.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.879+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70c0149440 con 0x7f70c00afd00 2024-04-27T07:51:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.879+0000 7f70b67fc640 1 -- 172.21.15.22:0/3486749346 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f70b00f1110 con 0x7f70c00afd00 2024-04-27T07:51:22.886 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 msgr2=0x7f70a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 0x7f70a406a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f70c0070450 tx=0x7f70bc008040 comp rx=0 tx=0).stop 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 msgr2=0x7f70c015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b0040 tx=0x7f70b001ac90 comp rx=0 tx=0).stop 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 shutdown_connections 2024-04-27T07:51:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c0161f30 0x7f70c00b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70c00afd00 0x7f70c015e0b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 --2- 172.21.15.22:0/3486749346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70c0071f70 0x7f70c0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 >> 172.21.15.22:0/3486749346 conn(0x7f70c00b21d0 msgr2=0x7f70c01567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 shutdown_connections 2024-04-27T07:51:22.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:22.883+0000 7f70c7ef8640 1 -- 172.21.15.22:0/3486749346 wait complete. 2024-04-27T07:51:22.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 Processor -- start 2024-04-27T07:51:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 -- start start 2024-04-27T07:51:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb500b4ee0 0x7feb500b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb500b5390 con 0x7feb500b3a10 2024-04-27T07:51:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb500b5670 con 0x7feb5015ecd0 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb5622c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7feb50073090 con 0x7feb500b4ee0 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7feb500b3a10 0x7feb500cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54154/0 (socket says 172.21.15.22:54154) 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb500b4ee0 0x7feb500b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4ffff640 1 -- 172.21.15.22:0/894643570 learned_addr learned my addr 172.21.15.22:0/894643570 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461022594 0 0) 0x7feb500b5390 con 0x7feb500b3a10 2024-04-27T07:51:23.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb38003530 con 0x7feb500b3a10 2024-04-27T07:51:23.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4effd640 1 -- 172.21.15.22:0/894643570 >> v1:172.21.15.183:6789/0 conn(0x7feb5015ecd0 legacy=0x7feb50072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4effd640 1 -- 172.21.15.22:0/894643570 >> v1:172.21.15.22:6789/0 conn(0x7feb500b3a10 legacy=0x7feb500cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1255005616 0 0) 0x7feb38003530 con 0x7feb500b3a10 2024-04-27T07:51:23.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.015+0000 7feb4effd640 1 -- 172.21.15.22:0/894643570 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb500ce340 con 0x7feb500b4ee0 2024-04-27T07:51:23.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb4effd640 1 --2- 172.21.15.22:0/894643570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb500b34d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb440097c0 tx=0x7feb4401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d40bdc315d4939 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440b1070 con 0x7feb500b4ee0 2024-04-27T07:51:23.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb440ac440 con 0x7feb500b4ee0 2024-04-27T07:51:23.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb4e7fc640 1 -- 172.21.15.22:0/894643570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48001a00 con 0x7feb500b4ee0 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- 172.21.15.22:0/894643570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 msgr2=0x7feb500b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 --2- 172.21.15.22:0/894643570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb500b34d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb440097c0 tx=0x7feb4401c390 comp rx=0 tx=0).stop 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- 172.21.15.22:0/894643570 shutdown_connections 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 --2- 172.21.15.22:0/894643570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb500b34d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- 172.21.15.22:0/894643570 >> 172.21.15.22:0/894643570 conn(0x7feb500b21d0 msgr2=0x7feb500b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- 172.21.15.22:0/894643570 shutdown_connections 2024-04-27T07:51:23.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- 172.21.15.22:0/894643570 wait complete. 2024-04-27T07:51:23.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 Processor -- start 2024-04-27T07:51:23.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.019+0000 7feb5622c640 1 -- start start 2024-04-27T07:51:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 0x7feb5015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb5015ecd0 0x7feb5014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb501696d0 con 0x7feb500b4ee0 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb50169550 con 0x7feb5015ecd0 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7feb50169850 con 0x7feb500b3a10 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58400/0 (socket says 172.21.15.22:58400) 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 -- 172.21.15.22:0/293551524 learned_addr learned my addr 172.21.15.22:0/293551524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4f7fe640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 0x7feb5015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 -- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 msgr2=0x7feb5015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 0x7feb5015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 -- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb5015ecd0 msgr2=0x7feb5014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb5015ecd0 0x7feb5014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 -- 172.21.15.22:0/293551524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5014b3f0 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4f7fe640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 0x7feb5015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4effd640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb440af040 tx=0x7feb44019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440b1040 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb440acd80 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5015b590 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44004220 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5014b970 con 0x7feb500b4ee0 2024-04-27T07:51:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5004ff10 con 0x7feb500b4ee0 2024-04-27T07:51:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feb440be020 con 0x7feb500b4ee0 2024-04-27T07:51:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 0x7feb3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feb44120090 con 0x7feb500b4ee0 2024-04-27T07:51:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.023+0000 7feb4f7fe640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 0x7feb3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.027+0000 7feb4f7fe640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 0x7feb3c06a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb40000f70 tx=0x7feb40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.027+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb44015fb0 con 0x7feb500b4ee0 2024-04-27T07:51:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb50167320 con 0x7feb500b4ee0 2024-04-27T07:51:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb37fff640 1 -- 172.21.15.22:0/293551524 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7feb440ab3a0 con 0x7feb500b4ee0 2024-04-27T07:51:23.192 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 msgr2=0x7feb3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 0x7feb3c06a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb40000f70 tx=0x7feb40009040 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 msgr2=0x7feb5014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb440af040 tx=0x7feb44019130 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 shutdown_connections 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7feb3c068380 0x7feb3c06a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb5015ecd0 0x7feb5014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb500b4ee0 0x7feb5014aa90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 --2- 172.21.15.22:0/293551524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb500b3a10 0x7feb5015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 >> 172.21.15.22:0/293551524 conn(0x7feb500b21d0 msgr2=0x7feb500b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 shutdown_connections 2024-04-27T07:51:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.187+0000 7feb5622c640 1 -- 172.21.15.22:0/293551524 wait complete. 2024-04-27T07:51:23.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 Processor -- start 2024-04-27T07:51:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 -- start start 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff3c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3c0b3e80 con 0x7eff3c0b30b0 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3c0cfbb0 con 0x7eff3c15aaa0 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff40c67640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c0ce7c0 con 0x7eff3c15ae80 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff3bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7eff3c0b30b0 0x7eff3c06dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54158/0 (socket says 172.21.15.22:54158) 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff3bfff640 1 -- 172.21.15.22:0/2374820218 learned_addr learned my addr 172.21.15.22:0/2374820218 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff3affd640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff3c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.323+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202443975 0 0) 0x7eff3c0b3e80 con 0x7eff3c0b30b0 2024-04-27T07:51:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff24003530 con 0x7eff3c0b30b0 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4029389638 0 0) 0x7eff24003530 con 0x7eff3c0b30b0 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 msgr2=0x7eff3c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff3c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 >> v1:172.21.15.183:6789/0 conn(0x7eff3c15aaa0 legacy=0x7eff3c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c0cfe90 con 0x7eff3c0b30b0 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7eff34001a00 con 0x7eff3c0b30b0 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 >> v1:172.21.15.22:6789/0 conn(0x7eff3c0b30b0 legacy=0x7eff3c06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240039f0 0x7eff24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff240043a0 0x7eff24004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff240050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0ce7c0 con 0x7eff240043a0 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400b9d0 con 0x7eff240039f0 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff2400bbd0 con 0x7eff3c15ae80 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff240043a0 0x7eff24004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3b7fe640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240039f0 0x7eff24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3bfff640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff240050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 -- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 msgr2=0x7eff240050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff240050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 -- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240039f0 msgr2=0x7eff24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240039f0 0x7eff24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 -- 172.21.15.22:0/2374820218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff24003530 con 0x7eff240043a0 2024-04-27T07:51:23.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3bfff640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c15ae80 0x7eff240050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3affd640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff240043a0 0x7eff24004870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff3000aac0 tx=0x7eff3001df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b6e34b892e0eab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff300b2070 con 0x7eff240043a0 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff300ad600 con 0x7eff240043a0 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3a7fc640 1 -- 172.21.15.22:0/2374820218 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000b040 con 0x7eff240043a0 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 -- 172.21.15.22:0/2374820218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff240043a0 msgr2=0x7eff24004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 --2- 172.21.15.22:0/2374820218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff240043a0 0x7eff24004870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff3000aac0 tx=0x7eff3001df80 comp rx=0 tx=0).stop 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff3bfff640 1 -- 172.21.15.22:0/2374820218 reap_dead start 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 -- 172.21.15.22:0/2374820218 shutdown_connections 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 -- 172.21.15.22:0/2374820218 >> 172.21.15.22:0/2374820218 conn(0x7eff3c06bda0 msgr2=0x7eff3c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 -- 172.21.15.22:0/2374820218 shutdown_connections 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.327+0000 7eff40c67640 1 -- 172.21.15.22:0/2374820218 wait complete. 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 Processor -- start 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- start start 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240043a0 0x7eff3c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c0cc8d0 0x7eff3c0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0cecf0 con 0x7eff3c073590 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0caf40 con 0x7eff240043a0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c0cf860 con 0x7eff3c0cc8d0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58410/0 (socket says 172.21.15.22:58410) 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 -- 172.21.15.22:0/3528507230 learned_addr learned my addr 172.21.15.22:0/3528507230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3bfff640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c0cc8d0 0x7eff3c0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3b7fe640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240043a0 0x7eff3c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 -- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c0cc8d0 msgr2=0x7eff3c0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c0cc8d0 0x7eff3c0721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 -- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240043a0 msgr2=0x7eff3c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240043a0 0x7eff3c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 -- 172.21.15.22:0/3528507230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c1f2b90 con 0x7eff3c073590 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff3affd640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff3001ddb0 tx=0x7eff3001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff300b2040 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff300b8470 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000b040 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c1f2e70 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c0c8850 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c04fea0 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.331+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff300b8840 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.335+0000 7eff23fff640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 0x7eff2806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.335+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7eff30121b10 con 0x7eff3c073590 2024-04-27T07:51:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.335+0000 7eff3b7fe640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 0x7eff2806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.335+0000 7eff3b7fe640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 0x7eff2806a870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0124b0 tx=0x7eff2c00a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.339+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff30017c90 con 0x7eff3c073590 2024-04-27T07:51:23.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.491+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff3c073a80 con 0x7eff3c073590 2024-04-27T07:51:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.491+0000 7eff23fff640 1 -- 172.21.15.22:0/3528507230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7eff300f1b10 con 0x7eff3c073590 2024-04-27T07:51:23.498 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 msgr2=0x7eff2806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 0x7eff2806a870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0124b0 tx=0x7eff2c00a9b0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 msgr2=0x7eff3c071c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff3001ddb0 tx=0x7eff3001ad20 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 shutdown_connections 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff280683b0 0x7eff2806a870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff3c0cc8d0 0x7eff3c0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff3c073590 0x7eff3c071c90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 --2- 172.21.15.22:0/3528507230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff240043a0 0x7eff3c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 >> 172.21.15.22:0/3528507230 conn(0x7eff3c06bda0 msgr2=0x7eff3c0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 shutdown_connections 2024-04-27T07:51:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.495+0000 7eff40c67640 1 -- 172.21.15.22:0/3528507230 wait complete. 2024-04-27T07:51:23.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 Processor -- start 2024-04-27T07:51:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 -- start start 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc406f870 0x7fefc40cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc40720d0 con 0x7fefc4144dd0 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc4072370 con 0x7fefc40cfc40 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc953d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefc4072650 con 0x7fefc406f870 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc2ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fefc4144dd0 0x7fefc406f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54170/0 (socket says 172.21.15.22:54170) 2024-04-27T07:51:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc2ffd640 1 -- 172.21.15.22:0/1747202723 learned_addr learned my addr 172.21.15.22:0/1747202723 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553464914 0 0) 0x7fefc40720d0 con 0x7fefc4144dd0 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefb0003530 con 0x7fefc4144dd0 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefba7fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc406f870 0x7fefc40cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3195507244 0 0) 0x7fefb0003530 con 0x7fefc4144dd0 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc406f870 msgr2=0x7fefc40cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc406f870 0x7fefc40cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 >> v1:172.21.15.183:6789/0 conn(0x7fefc40cfc40 legacy=0x7fefc40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc4072930 con 0x7fefc4144dd0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fefb4001680 con 0x7fefc4144dd0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 >> v1:172.21.15.22:6789/0 conn(0x7fefc4144dd0 legacy=0x7fefc406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00039f0 0x7fefb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefb0004810 0x7fefb0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb00051d0 0x7fefb000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4072650 con 0x7fefb00051d0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefb000c0a0 con 0x7fefb00039f0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefb000c2a0 con 0x7fefb0004810 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefba7fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc406f870 0x7fefc40cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb00051d0 0x7fefb000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefba7fc640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefb0004810 0x7fefb0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc2ffd640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00039f0 0x7fefb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 -- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefb0004810 msgr2=0x7fefb0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefb0004810 0x7fefb0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 -- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00039f0 msgr2=0x7fefb0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00039f0 0x7fefb0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc37fe640 1 -- 172.21.15.22:0/1747202723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0003530 con 0x7fefb00051d0 2024-04-27T07:51:23.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.623+0000 7fefc2ffd640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00039f0 0x7fefb0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc37fe640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb00051d0 0x7fefb000b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefbc001e70 tx=0x7fefbc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378a8598706863da server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc001a90 con 0x7fefb00051d0 2024-04-27T07:51:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc003b80 con 0x7fefb00051d0 2024-04-27T07:51:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc27fc640 1 -- 172.21.15.22:0/1747202723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc005080 con 0x7fefb00051d0 2024-04-27T07:51:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- 172.21.15.22:0/1747202723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb00051d0 msgr2=0x7fefb000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 --2- 172.21.15.22:0/1747202723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb00051d0 0x7fefb000b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefbc001e70 tx=0x7fefbc01c780 comp rx=0 tx=0).stop 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc37fe640 1 -- 172.21.15.22:0/1747202723 reap_dead start 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- 172.21.15.22:0/1747202723 shutdown_connections 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- 172.21.15.22:0/1747202723 >> 172.21.15.22:0/1747202723 conn(0x7fefc4067f10 msgr2=0x7fefc406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- 172.21.15.22:0/1747202723 shutdown_connections 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- 172.21.15.22:0/1747202723 wait complete. 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 Processor -- start 2024-04-27T07:51:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.627+0000 7fefc953d640 1 -- start start 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00051d0 0x7fefc40a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc4140e50 0x7fefc413b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40d00b0 con 0x7fefc413e180 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41451b0 con 0x7fefb00051d0 2024-04-27T07:51:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40d0230 con 0x7fefc4140e50 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58422/0 (socket says 172.21.15.22:58422) 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 -- 172.21.15.22:0/2401691079 learned_addr learned my addr 172.21.15.22:0/2401691079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc2ffd640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00051d0 0x7fefc40a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefba7fc640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc4140e50 0x7fefc413b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 -- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc4140e50 msgr2=0x7fefc413b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc4140e50 0x7fefc413b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 -- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00051d0 msgr2=0x7fefc40a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00051d0 0x7fefc40a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 -- 172.21.15.22:0/2401691079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40ad270 con 0x7fefc413e180 2024-04-27T07:51:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc37fe640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefbc01ccf0 tx=0x7fefb0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc001a90 con 0x7fefc413e180 2024-04-27T07:51:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc0b8070 con 0x7fefc413e180 2024-04-27T07:51:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc40a9ce0 con 0x7fefc413e180 2024-04-27T07:51:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc007dc0 con 0x7fefc413e180 2024-04-27T07:51:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc40aa2a0 con 0x7fefc413e180 2024-04-27T07:51:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc40ad440 con 0x7fefc413e180 2024-04-27T07:51:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fefbc0a6480 con 0x7fefc413e180 2024-04-27T07:51:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 0x7fefa406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fefbc1217a0 con 0x7fefc413e180 2024-04-27T07:51:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.631+0000 7fefc2ffd640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 0x7fefa406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.635+0000 7fefc2ffd640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 0x7fefa406a870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c10 tx=0x7fefb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.635+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fefbc0ac020 con 0x7fefc413e180 2024-04-27T07:51:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.795+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefc4068c30 con 0x7fefc413e180 2024-04-27T07:51:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.795+0000 7fefbbfff640 1 -- 172.21.15.22:0/2401691079 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fefbc016490 con 0x7fefc413e180 2024-04-27T07:51:23.801 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:23.801 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 msgr2=0x7fefa406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 0x7fefa406a870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c10 tx=0x7fefb4009040 comp rx=0 tx=0).stop 2024-04-27T07:51:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 msgr2=0x7fefc413e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefbc01ccf0 tx=0x7fefb0005840 comp rx=0 tx=0).stop 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 shutdown_connections 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fefa40683b0 0x7fefa406a870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc413e180 0x7fefc413e600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefc4140e50 0x7fefc413b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 --2- 172.21.15.22:0/2401691079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefb00051d0 0x7fefc40a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 >> 172.21.15.22:0/2401691079 conn(0x7fefc4067f10 msgr2=0x7fefc415e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 shutdown_connections 2024-04-27T07:51:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.799+0000 7fefc953d640 1 -- 172.21.15.22:0/2401691079 wait complete. 2024-04-27T07:51:23.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:23.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:23.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 Processor -- start 2024-04-27T07:51:23.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 -- start start 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f744c15ecd0 0x7f744c072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b4ee0 0x7f744c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f744c0b3a10 0x7f744c0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f744c0b5390 con 0x7f744c15ecd0 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f744c0b5670 con 0x7f744c0b3a10 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f74509be640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f744c0ce340 con 0x7f744c0b4ee0 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f744c15ecd0 0x7f744c072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f744c15ecd0 0x7f744c072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58430/0 (socket says 172.21.15.22:58430) 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 -- 172.21.15.22:0/1897832499 learned_addr learned my addr 172.21.15.22:0/1897832499 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f7449d9b640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b4ee0 0x7f744c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 -- 172.21.15.22:0/1897832499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b4ee0 msgr2=0x7f744c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b4ee0 0x7f744c0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 -- 172.21.15.22:0/1897832499 >> v2:172.21.15.183:3300/0 conn(0x7f744c0b3a10 msgr2=0x7f744c0cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744ad9d640 1 --2- 172.21.15.22:0/1897832499 >> v2:172.21.15.183:3300/0 conn(0x7f744c0b3a10 0x7f744c0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 --2- 172.21.15.22:0/1897832499 >> v2:172.21.15.183:3300/0 conn(0x7f744c0b3a10 0x7f744c0cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.931+0000 7f744a59c640 1 -- 172.21.15.22:0/1897832499 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c0ce620 con 0x7f744c15ecd0 2024-04-27T07:51:23.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744a59c640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c15ecd0 0x7f744c072980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f743c009870 tx=0x7f743c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce92ef3266e3607b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744959a640 1 -- 172.21.15.22:0/1897832499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0b1070 con 0x7f744c15ecd0 2024-04-27T07:51:23.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744959a640 1 -- 172.21.15.22:0/1897832499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c0ac620 con 0x7f744c15ecd0 2024-04-27T07:51:23.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744959a640 1 -- 172.21.15.22:0/1897832499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0ac9f0 con 0x7f744c15ecd0 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1897832499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c15ecd0 msgr2=0x7f744c072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c15ecd0 0x7f744c072980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f743c009870 tx=0x7f743c01c4f0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1897832499 shutdown_connections 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- 172.21.15.22:0/1897832499 >> v2:172.21.15.183:3300/0 conn(0x7f744c0b3a10 0x7f744c0cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b4ee0 0x7f744c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- 172.21.15.22:0/1897832499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c15ecd0 0x7f744c072980 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1897832499 >> 172.21.15.22:0/1897832499 conn(0x7f744c0b21d0 msgr2=0x7f744c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1897832499 shutdown_connections 2024-04-27T07:51:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1897832499 wait complete. 2024-04-27T07:51:23.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 Processor -- start 2024-04-27T07:51:23.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- start start 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 0x7f744c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 0x7f744c163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f744c15f550 con 0x7f744c0b4ee0 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f744c15f3d0 con 0x7f744c15ecd0 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f744c15f6d0 con 0x7f744c0b3a10 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58444/0 (socket says 172.21.15.22:58444) 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 -- 172.21.15.22:0/1266720761 learned_addr learned my addr 172.21.15.22:0/1266720761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744a59c640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 0x7f744c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 -- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 msgr2=0x7f744c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 0x7f744c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744ad9d640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 0x7f744c163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 -- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 msgr2=0x7f744c163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 0x7f744c163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 -- 172.21.15.22:0/1266720761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c164650 con 0x7f744c0b4ee0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744a59c640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 0x7f744c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f744ad9d640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 0x7f744c163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f7449d9b640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f744000c7f0 tx=0x7f744000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440015070 con 0x7f744c0b4ee0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440004220 con 0x7f744c0b4ee0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440003620 con 0x7f744c0b4ee0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c164930 con 0x7f744c0b4ee0 2024-04-27T07:51:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.935+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c1f6450 con 0x7f744c0b4ee0 2024-04-27T07:51:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744c04ff10 con 0x7f744c0b4ee0 2024-04-27T07:51:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74400039d0 con 0x7f744c0b4ee0 2024-04-27T07:51:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f743affd640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 0x7f742c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7440086a50 con 0x7f744c0b4ee0 2024-04-27T07:51:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f744a59c640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 0x7f742c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.939+0000 7f744a59c640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 0x7f742c06a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f743c0ae000 tx=0x7f743c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:23.943+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7440057010 con 0x7f744c0b4ee0 2024-04-27T07:51:24.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.099+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f744c153ac0 con 0x7f744c0b4ee0 2024-04-27T07:51:24.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.099+0000 7f743affd640 1 -- 172.21.15.22:0/1266720761 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7440056910 con 0x7f744c0b4ee0 2024-04-27T07:51:24.106 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:24.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 msgr2=0x7f742c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 0x7f742c06a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f743c0ae000 tx=0x7f743c008040 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 msgr2=0x7f744c152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f744000c7f0 tx=0x7f744000ccb0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 shutdown_connections 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f742c068380 0x7f742c06a840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c15ecd0 0x7f744c163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f744c0b4ee0 0x7f744c152480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 --2- 172.21.15.22:0/1266720761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f744c0b3a10 0x7f744c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 >> 172.21.15.22:0/1266720761 conn(0x7f744c0b21d0 msgr2=0x7f744c1632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 shutdown_connections 2024-04-27T07:51:24.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.103+0000 7f74509be640 1 -- 172.21.15.22:0/1266720761 wait complete. 2024-04-27T07:51:24.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.243+0000 7f34a5b6a640 1 Processor -- start 2024-04-27T07:51:24.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f34a5b6a640 1 -- start start 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f34a5b6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f34a0072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f34a5b6a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a00b5390 con 0x7f34a00b4ee0 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f34a5b6a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a00b5670 con 0x7f34a00cee20 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f34a5b6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f34a00ce340 con 0x7f34a015ecd0 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f3496ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f34a00b4ee0 0x7f34a00b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54172/0 (socket says 172.21.15.22:54172) 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f3496ffd640 1 -- 172.21.15.22:0/1094335130 learned_addr learned my addr 172.21.15.22:0/1094335130 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f34a0072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994478394 0 0) 0x7f34a00b5390 con 0x7f34a00b4ee0 2024-04-27T07:51:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f348c003530 con 0x7f34a00b4ee0 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3290507529 0 0) 0x7f348c003530 con 0x7f34a00b4ee0 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 msgr2=0x7f34a0072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f34a0072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 >> v1:172.21.15.183:6789/0 conn(0x7f34a00cee20 legacy=0x7f34a006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00ce620 con 0x7f34a00b4ee0 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f3488001680 con 0x7f34a00b4ee0 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 >> v1:172.21.15.22:6789/0 conn(0x7f34a00b4ee0 legacy=0x7f34a00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c0039f0 0x7f348c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f348c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c004d50 0x7f348c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00ce340 con 0x7f348c004d50 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f348c00ba80 con 0x7f348c0039f0 2024-04-27T07:51:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f348c00bc80 con 0x7f34a015ecd0 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c004d50 0x7f348c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f3496ffd640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f348c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c0039f0 0x7f348c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 -- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 msgr2=0x7f348c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a015ecd0 0x7f348c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 -- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c0039f0 msgr2=0x7f348c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c0039f0 0x7f348c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.247+0000 7f349ffff640 1 -- 172.21.15.22:0/1094335130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c003530 con 0x7f348c004d50 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349ffff640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c004d50 0x7f348c0051d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3498001e40 tx=0x7f349801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437342707a5211ce server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498001a90 con 0x7f348c004d50 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3498003b80 con 0x7f348c004d50 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349effd640 1 -- 172.21.15.22:0/1094335130 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498005080 con 0x7f348c004d50 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1094335130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c004d50 msgr2=0x7f348c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1094335130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c004d50 0x7f348c0051d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3498001e40 tx=0x7f349801c760 comp rx=0 tx=0).stop 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349ffff640 1 -- 172.21.15.22:0/1094335130 reap_dead start 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1094335130 shutdown_connections 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1094335130 >> 172.21.15.22:0/1094335130 conn(0x7f34a00b21d0 msgr2=0x7f34a00b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1094335130 shutdown_connections 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1094335130 wait complete. 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 Processor -- start 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- start start 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 0x7f34a00c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a00c79f0 0x7f34a015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00b3be0 con 0x7f34a00a9490 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a006f7f0 con 0x7f348c004d50 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f34a5b6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f34a00c8a70 con 0x7f34a00c79f0 2024-04-27T07:51:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58456/0 (socket says 172.21.15.22:58456) 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.251+0000 7f349ffff640 1 -- 172.21.15.22:0/1963914740 learned_addr learned my addr 172.21.15.22:0/1963914740 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 0x7f34a00c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 -- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a00c79f0 msgr2=0x7f34a015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a00c79f0 0x7f34a015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 -- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 msgr2=0x7f34a00c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 0x7f34a00c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 -- 172.21.15.22:0/1963914740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00c7ef0 con 0x7f34a00a9490 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 0x7f34a00c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349ffff640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3498098170 tx=0x7f348c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980ba070 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3498006b60 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498001a90 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a00c81d0 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a00c8790 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a004ff10 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3498007030 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 0x7f348006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3498120030 con 0x7f34a00a9490 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 0x7f348006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.255+0000 7f349f7fe640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 0x7f348006a870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3490009a20 tx=0x7f3490008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.259+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34980ac020 con 0x7f34a00a9490 2024-04-27T07:51:24.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.423+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34a00cfc60 con 0x7f34a00a9490 2024-04-27T07:51:24.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.423+0000 7f349cff9640 1 -- 172.21.15.22:0/1963914740 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f3498016470 con 0x7f34a00a9490 2024-04-27T07:51:24.429 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:24.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 msgr2=0x7f348006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 0x7f348006a870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3490009a20 tx=0x7f3490008040 comp rx=0 tx=0).stop 2024-04-27T07:51:24.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 msgr2=0x7f34a00a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3498098170 tx=0x7f348c005840 comp rx=0 tx=0).stop 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 shutdown_connections 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f34800683b0 0x7f348006a870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34a00a9490 0x7f34a00a9940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34a00c79f0 0x7f34a015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 --2- 172.21.15.22:0/1963914740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f348c004d50 0x7f34a00c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 >> 172.21.15.22:0/1963914740 conn(0x7f34a00b21d0 msgr2=0x7f34a015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 shutdown_connections 2024-04-27T07:51:24.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.427+0000 7f34a5b6a640 1 -- 172.21.15.22:0/1963914740 wait complete. 2024-04-27T07:51:24.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 Processor -- start 2024-04-27T07:51:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 -- start start 2024-04-27T07:51:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61f4164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f40cdb10 con 0x7f61f406f750 2024-04-27T07:51:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f40cddf0 con 0x7f61f4071f70 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61fa5ea640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f61f40ce0d0 con 0x7f61f40cffe0 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f8b87640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61f406f750 0x7f61f41668e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54180/0 (socket says 172.21.15.22:54180) 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f8b87640 1 -- 172.21.15.22:0/3276080378 learned_addr learned my addr 172.21.15.22:0/3276080378 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61f4164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592465321 0 0) 0x7f61f40cdb10 con 0x7f61f406f750 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0003530 con 0x7f61f406f750 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1388996096 0 0) 0x7f61e0003530 con 0x7f61f406f750 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 msgr2=0x7f61f4164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61f4164390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 >> v1:172.21.15.183:6789/0 conn(0x7f61f4071f70 legacy=0x7f61f4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f406e220 con 0x7f61f406f750 2024-04-27T07:51:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f61ec001680 con 0x7f61f406f750 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 >> v1:172.21.15.22:6789/0 conn(0x7f61f406f750 legacy=0x7f61f41668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61e00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0003f70 0x7f61e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40ce0d0 con 0x7f61e0003f70 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61e000bb60 con 0x7f61e0004d50 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f61e000bd60 con 0x7f61f40cffe0 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0003f70 0x7f61e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f3fff640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61e00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 -- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 msgr2=0x7f61e00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61f40cffe0 0x7f61e00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 -- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61e0004d50 msgr2=0x7f61e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.567+0000 7f61f37fe640 1 -- 172.21.15.22:0/3276080378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0003530 con 0x7f61e0003f70 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61f37fe640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0003f70 0x7f61e0004810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009fc0 tx=0x7f61e401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07740b9a89af808 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b2070 con 0x7f61e0003f70 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e40a5a40 con 0x7f61e0003f70 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61f2ffd640 1 -- 172.21.15.22:0/3276080378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4009c10 con 0x7f61e0003f70 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- 172.21.15.22:0/3276080378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0003f70 msgr2=0x7f61e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/3276080378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0003f70 0x7f61e0004810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009fc0 tx=0x7f61e401cc80 comp rx=0 tx=0).stop 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61f8b87640 1 -- 172.21.15.22:0/3276080378 reap_dead start 2024-04-27T07:51:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- 172.21.15.22:0/3276080378 shutdown_connections 2024-04-27T07:51:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- 172.21.15.22:0/3276080378 >> 172.21.15.22:0/3276080378 conn(0x7f61f40b21d0 msgr2=0x7f61f40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- 172.21.15.22:0/3276080378 shutdown_connections 2024-04-27T07:51:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- 172.21.15.22:0/3276080378 wait complete. 2024-04-27T07:51:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 Processor -- start 2024-04-27T07:51:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.571+0000 7f61fa5ea640 1 -- start start 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61e0003f70 0x7f61f4145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61f415e220 0x7f61f413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40cd8c0 con 0x7f61f4167750 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40cd0c0 con 0x7f61f415e220 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f61f406dfd0 con 0x7f61e0003f70 2024-04-27T07:51:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58472/0 (socket says 172.21.15.22:58472) 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 -- 172.21.15.22:0/571829780 learned_addr learned my addr 172.21.15.22:0/571829780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f8b87640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61f415e220 0x7f61f413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f3fff640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61e0003f70 0x7f61f4145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 -- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61e0003f70 msgr2=0x7f61f4145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61e0003f70 0x7f61f4145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 -- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61f415e220 msgr2=0x7f61f413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61f415e220 0x7f61f413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 -- 172.21.15.22:0/571829780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f415c6e0 con 0x7f61f4167750 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f37fe640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61e4098560 tx=0x7f61e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b2050 con 0x7f61f4167750 2024-04-27T07:51:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e40b3040 con 0x7f61f4167750 2024-04-27T07:51:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4003bc0 con 0x7f61f4167750 2024-04-27T07:51:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f40b4910 con 0x7f61f4167750 2024-04-27T07:51:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f415da20 con 0x7f61f4167750 2024-04-27T07:51:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f404ff10 con 0x7f61f4167750 2024-04-27T07:51:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f61e40c0030 con 0x7f61f4167750 2024-04-27T07:51:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 0x7f61d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61e41212c0 con 0x7f61f4167750 2024-04-27T07:51:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.575+0000 7f61f3fff640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 0x7f61d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.579+0000 7f61f3fff640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 0x7f61d406a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c10 tx=0x7f61dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.579+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f61e40c7020 con 0x7f61f4167750 2024-04-27T07:51:24.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.739+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61f4072cb0 con 0x7f61f4167750 2024-04-27T07:51:24.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.739+0000 7f61f0ff9640 1 -- 172.21.15.22:0/571829780 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f61e4016990 con 0x7f61f4167750 2024-04-27T07:51:24.745 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:24.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 msgr2=0x7f61d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 0x7f61d406a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c10 tx=0x7f61dc009040 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 msgr2=0x7f61f415c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61e4098560 tx=0x7f61e0005840 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 shutdown_connections 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f61d4068380 0x7f61d406a840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61f415e220 0x7f61f413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f4167750 0x7f61f415c1a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 --2- 172.21.15.22:0/571829780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61e0003f70 0x7f61f4145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 >> 172.21.15.22:0/571829780 conn(0x7f61f40b21d0 msgr2=0x7f61f40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 shutdown_connections 2024-04-27T07:51:24.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.743+0000 7f61fa5ea640 1 -- 172.21.15.22:0/571829780 wait complete. 2024-04-27T07:51:24.781 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-04-27T07:51:24.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:24.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.887+0000 7fa7e8abc640 1 Processor -- start 2024-04-27T07:51:24.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.887+0000 7fa7e8abc640 1 -- start start 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e8abc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7e41623a0 0x7fa7e4162800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e8abc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa7e406f560 0x7fa7e406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e8abc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e4145220 con 0x7fa7e4144e40 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e8abc640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e406ff10 con 0x7fa7e41623a0 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e8abc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e406f0e0 con 0x7fa7e406f560 2024-04-27T07:51:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7e41623a0 0x7fa7e4162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7e41623a0 0x7fa7e4162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58486/0 (socket says 172.21.15.22:58486) 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 -- 172.21.15.22:0/2394574373 learned_addr learned my addr 172.21.15.22:0/2394574373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e2d9d640 1 --2- 172.21.15.22:0/2394574373 >> v2:172.21.15.183:3300/0 conn(0x7fa7e406f560 0x7fa7e406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 -- 172.21.15.22:0/2394574373 >> v2:172.21.15.183:3300/0 conn(0x7fa7e406f560 msgr2=0x7fa7e406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 --2- 172.21.15.22:0/2394574373 >> v2:172.21.15.183:3300/0 conn(0x7fa7e406f560 0x7fa7e406f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 -- 172.21.15.22:0/2394574373 >> v1:172.21.15.183:6789/0 conn(0x7fa7e4144e40 legacy=0x7fa7e406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 -- 172.21.15.22:0/2394574373 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e4162df0 con 0x7fa7e41623a0 2024-04-27T07:51:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7d9d9b640 1 --2- 172.21.15.22:0/2394574373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e41623a0 0x7fa7e4162800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40732b0 tx=0x7fa7d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450893c9581be082 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e1d9b640 1 -- 172.21.15.22:0/2394574373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40aa680 con 0x7fa7e41623a0 2024-04-27T07:51:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e1d9b640 1 -- 172.21.15.22:0/2394574373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40af070 con 0x7fa7e41623a0 2024-04-27T07:51:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.891+0000 7fa7e1d9b640 1 -- 172.21.15.22:0/2394574373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40aaa40 con 0x7fa7e41623a0 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2394574373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e41623a0 msgr2=0x7fa7e4162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2394574373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e41623a0 0x7fa7e4162800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40732b0 tx=0x7fa7d401c520 comp rx=0 tx=0).stop 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2394574373 shutdown_connections 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2394574373 >> v2:172.21.15.183:3300/0 conn(0x7fa7e406f560 0x7fa7e406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2394574373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e41623a0 0x7fa7e4162800 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2394574373 >> 172.21.15.22:0/2394574373 conn(0x7fa7e4067f10 msgr2=0x7fa7e406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2394574373 shutdown_connections 2024-04-27T07:51:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2394574373 wait complete. 2024-04-27T07:51:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 Processor -- start 2024-04-27T07:51:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- start start 2024-04-27T07:51:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7e4144e40 0x7fa7e4141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e41623a0 0x7fa7e40b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4072370 con 0x7fa7e406f560 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40721f0 con 0x7fa7e41623a0 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e40af460 con 0x7fa7e4144e40 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58490/0 (socket says 172.21.15.22:58490) 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 -- 172.21.15.22:0/2880629026 learned_addr learned my addr 172.21.15.22:0/2880629026 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e2d9d640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e41623a0 0x7fa7e40b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7d9d9b640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7e4144e40 0x7fa7e4141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 -- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7e4144e40 msgr2=0x7fa7e4141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7e4144e40 0x7fa7e4141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 -- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e41623a0 msgr2=0x7fa7e40b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e41623a0 0x7fa7e40b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 -- 172.21.15.22:0/2880629026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e415c860 con 0x7fa7e406f560 2024-04-27T07:51:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e259c640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00c420 tx=0x7fa7cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0052d0 con 0x7fa7e406f560 2024-04-27T07:51:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc005470 con 0x7fa7e406f560 2024-04-27T07:51:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e415cb40 con 0x7fa7e406f560 2024-04-27T07:51:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc007260 con 0x7fa7e406f560 2024-04-27T07:51:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.895+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e40cea50 con 0x7fa7e406f560 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e4068c30 con 0x7fa7e406f560 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa7cc007400 con 0x7fa7e406f560 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7db7fe640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 0x7fa7c406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7cc085f20 con 0x7fa7e406f560 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7d9d9b640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 0x7fa7c406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.899+0000 7fa7d9d9b640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 0x7fa7c406a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ae000 tx=0x7fa7d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:24.903+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa7e4145220 con 0x7fa7e406f560 2024-04-27T07:51:25.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.063+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7e4142080 con 0x7fa7e406f560 2024-04-27T07:51:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7db7fe640 1 -- 172.21.15.22:0/2880629026 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fa7cc056440 con 0x7fa7e406f560 2024-04-27T07:51:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:25.072 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 msgr2=0x7fa7c406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 0x7fa7c406a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ae000 tx=0x7fa7d4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 msgr2=0x7fa7e40af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00c420 tx=0x7fa7cc00c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 shutdown_connections 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa7c40683b0 0x7fa7c406a870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e41623a0 0x7fa7e40b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7e4144e40 0x7fa7e4141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 --2- 172.21.15.22:0/2880629026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e406f560 0x7fa7e40af820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 >> 172.21.15.22:0/2880629026 conn(0x7fa7e4067f10 msgr2=0x7fa7e4168d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 shutdown_connections 2024-04-27T07:51:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.067+0000 7fa7e8abc640 1 -- 172.21.15.22:0/2880629026 wait complete. 2024-04-27T07:51:25.089 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:25.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:25.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 Processor -- start 2024-04-27T07:51:25.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 -- start start 2024-04-27T07:51:25.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6eb809ae40 0x7f6eb815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6eb80b37a0 0x7f6eb80d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb80ce240 con 0x7f6eb806bbd0 2024-04-27T07:51:25.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb80cedc0 con 0x7f6eb80b37a0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6ebdf51640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb80ce520 con 0x7f6eb809ae40 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb77fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6eb806bbd0 0x7f6eb806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54184/0 (socket says 172.21.15.22:54184) 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb77fe640 1 -- 172.21.15.22:0/76398150 learned_addr learned my addr 172.21.15.22:0/76398150 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb7fff640 1 --2- 172.21.15.22:0/76398150 >> v2:172.21.15.22:3300/0 conn(0x7f6eb80b37a0 0x7f6eb80d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb6ffd640 1 --2- 172.21.15.22:0/76398150 >> v2:172.21.15.183:3300/0 conn(0x7f6eb809ae40 0x7f6eb815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007912300 0 0) 0x7f6eb80ce240 con 0x7f6eb806bbd0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ea0003530 con 0x7f6eb806bbd0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb7fff640 1 -- 172.21.15.22:0/76398150 >> v2:172.21.15.183:3300/0 conn(0x7f6eb809ae40 msgr2=0x7f6eb815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 648228893 0 0) 0x7f6ea0003530 con 0x7f6eb806bbd0 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb7fff640 1 --2- 172.21.15.22:0/76398150 >> v2:172.21.15.183:3300/0 conn(0x7f6eb809ae40 0x7f6eb815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb7fff640 1 -- 172.21.15.22:0/76398150 >> v1:172.21.15.22:6789/0 conn(0x7f6eb806bbd0 legacy=0x7f6eb806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.207+0000 7f6eb7fff640 1 -- 172.21.15.22:0/76398150 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb80ce800 con 0x7f6eb80b37a0 2024-04-27T07:51:25.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb7fff640 1 --2- 172.21.15.22:0/76398150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb80b37a0 0x7f6eb80d0690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b0a70 tx=0x7f6ea801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a36de3307aa866 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80b0070 con 0x7f6eb80b37a0 2024-04-27T07:51:25.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea80ab400 con 0x7f6eb80b37a0 2024-04-27T07:51:25.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb67fc640 1 -- 172.21.15.22:0/76398150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0001a00 con 0x7f6eb80b37a0 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- 172.21.15.22:0/76398150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb80b37a0 msgr2=0x7f6eb80d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/76398150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb80b37a0 0x7f6eb80d0690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b0a70 tx=0x7f6ea801c370 comp rx=0 tx=0).stop 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- 172.21.15.22:0/76398150 shutdown_connections 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/76398150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb80b37a0 0x7f6eb80d0690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/76398150 >> v2:172.21.15.183:3300/0 conn(0x7f6eb809ae40 0x7f6eb815cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- 172.21.15.22:0/76398150 >> 172.21.15.22:0/76398150 conn(0x7f6eb809aa30 msgr2=0x7f6eb80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- 172.21.15.22:0/76398150 shutdown_connections 2024-04-27T07:51:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- 172.21.15.22:0/76398150 wait complete. 2024-04-27T07:51:25.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 Processor -- start 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- start start 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6eb806bbd0 0x7f6eb81524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6eb80b37a0 0x7f6eb8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb815d370 con 0x7f6eb809ae40 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb80d11f0 con 0x7f6eb80b37a0 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6ebdf51640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb8169b10 con 0x7f6eb806bbd0 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58508/0 (socket says 172.21.15.22:58508) 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb6ffd640 1 -- 172.21.15.22:0/2126543883 learned_addr learned my addr 172.21.15.22:0/2126543883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.211+0000 7f6eb7fff640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6eb80b37a0 0x7f6eb8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 -- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6eb806bbd0 msgr2=0x7f6eb81524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6eb806bbd0 0x7f6eb81524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 -- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6eb80b37a0 msgr2=0x7f6eb8159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6eb80b37a0 0x7f6eb8159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 -- 172.21.15.22:0/2126543883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb8152a30 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb6ffd640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e930 tx=0x7f6eac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac017070 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eac00ca00 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00cdf0 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb8153d70 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb8153270 con 0x7f6eb809ae40 2024-04-27T07:51:25.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb804ff10 con 0x7f6eb809ae40 2024-04-27T07:51:25.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6eac01a070 con 0x7f6eb809ae40 2024-04-27T07:51:25.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 0x7f6ea406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6eac086920 con 0x7f6eb809ae40 2024-04-27T07:51:25.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb77fe640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 0x7f6ea406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.215+0000 7f6eb77fe640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 0x7f6ea406a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0002240 tx=0x7f6eb0001dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.219+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6eb8152a30 con 0x7f6eb809ae40 2024-04-27T07:51:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.375+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6eb80b14c0 con 0x7f6eb809ae40 2024-04-27T07:51:25.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.375+0000 7f6e9ffff640 1 -- 172.21.15.22:0/2126543883 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6eac057030 con 0x7f6eb809ae40 2024-04-27T07:51:25.382 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 msgr2=0x7f6ea406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 0x7f6ea406a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0002240 tx=0x7f6eb0001dd0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 msgr2=0x7f6eb8153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e930 tx=0x7f6eac00edf0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 shutdown_connections 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ea4068380 0x7f6ea406a840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6eb80b37a0 0x7f6eb8159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6eb809ae40 0x7f6eb8153830 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 --2- 172.21.15.22:0/2126543883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6eb806bbd0 0x7f6eb81524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 >> 172.21.15.22:0/2126543883 conn(0x7f6eb809aa30 msgr2=0x7f6eb80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 shutdown_connections 2024-04-27T07:51:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.379+0000 7f6ebdf51640 1 -- 172.21.15.22:0/2126543883 wait complete. 2024-04-27T07:51:25.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:25.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 Processor -- start 2024-04-27T07:51:25.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 -- start start 2024-04-27T07:51:25.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0580cfd90 0x7fd0580af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580709a0 0x7fd0580afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0580b3940 con 0x7fd05815ecb0 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580af270 con 0x7fd0580cfd90 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05e95d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580aef00 con 0x7fd0580709a0 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0580cfd90 0x7fd0580af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0580cfd90 0x7fd0580af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58514/0 (socket says 172.21.15.22:58514) 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 -- 172.21.15.22:0/1904838881 learned_addr learned my addr 172.21.15.22:0/1904838881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd05cefa640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580709a0 0x7fd0580afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 -- 172.21.15.22:0/1904838881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580709a0 msgr2=0x7fd0580afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580709a0 0x7fd0580afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 -- 172.21.15.22:0/1904838881 >> v1:172.21.15.183:6789/0 conn(0x7fd05815ecb0 legacy=0x7fd0580cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.515+0000 7fd0577fe640 1 -- 172.21.15.22:0/1904838881 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058070430 con 0x7fd0580cfd90 2024-04-27T07:51:25.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd0577fe640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0580cfd90 0x7fd0580af700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd04800d180 tx=0x7fd04801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82916cd628f1db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd056ffd640 1 -- 172.21.15.22:0/1904838881 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800b2d0 con 0x7fd0580cfd90 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd056ffd640 1 -- 172.21.15.22:0/1904838881 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04800b470 con 0x7fd0580cfd90 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd056ffd640 1 -- 172.21.15.22:0/1904838881 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480a7a70 con 0x7fd0580cfd90 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- 172.21.15.22:0/1904838881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0580cfd90 msgr2=0x7fd0580af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0580cfd90 0x7fd0580af700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd04800d180 tx=0x7fd04801eb80 comp rx=0 tx=0).stop 2024-04-27T07:51:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- 172.21.15.22:0/1904838881 shutdown_connections 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580709a0 0x7fd0580afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- 172.21.15.22:0/1904838881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0580cfd90 0x7fd0580af700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- 172.21.15.22:0/1904838881 >> 172.21.15.22:0/1904838881 conn(0x7fd05809aa30 msgr2=0x7fd0580b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- 172.21.15.22:0/1904838881 shutdown_connections 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- 172.21.15.22:0/1904838881 wait complete. 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 Processor -- start 2024-04-27T07:51:25.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- start start 2024-04-27T07:51:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0580709a0 0x7fd0580b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580cfd90 0x7fd0580b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd05806bf60 con 0x7fd05815ecb0 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd05806bde0 con 0x7fd0580709a0 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05e95d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd05806c0e0 con 0x7fd0580cfd90 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05cefa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05cefa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58524/0 (socket says 172.21.15.22:58524) 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.519+0000 7fd05cefa640 1 -- 172.21.15.22:0/254249827 learned_addr learned my addr 172.21.15.22:0/254249827 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd057fff640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0580709a0 0x7fd0580b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd0577fe640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580cfd90 0x7fd0580b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 -- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580cfd90 msgr2=0x7fd0580b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580cfd90 0x7fd0580b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 -- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0580709a0 msgr2=0x7fd0580b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0580709a0 0x7fd0580b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 -- 172.21.15.22:0/254249827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580b40b0 con 0x7fd05815ecb0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05cefa640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d600 tx=0x7fd05000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500125f0 con 0x7fd05815ecb0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050019070 con 0x7fd05815ecb0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0580b4390 con 0x7fd05815ecb0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500129f0 con 0x7fd05815ecb0 2024-04-27T07:51:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0580b4950 con 0x7fd05815ecb0 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05804fef0 con 0x7fd05815ecb0 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd050012b90 con 0x7fd05815ecb0 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 0x7fd03806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd050086db0 con 0x7fd05815ecb0 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd057fff640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 0x7fd03806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.523+0000 7fd057fff640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 0x7fd03806a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0400015b0 tx=0x7fd040009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.527+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd050057400 con 0x7fd05815ecb0 2024-04-27T07:51:25.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.683+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd05806e0f0 con 0x7fd05815ecb0 2024-04-27T07:51:25.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.683+0000 7fd054ff9640 1 -- 172.21.15.22:0/254249827 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd050056d00 con 0x7fd05815ecb0 2024-04-27T07:51:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 msgr2=0x7fd03806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 0x7fd03806a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0400015b0 tx=0x7fd040009040 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 msgr2=0x7fd0580c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d600 tx=0x7fd05000dac0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 shutdown_connections 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd038068380 0x7fd03806a840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd05815ecb0 0x7fd0580c58a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0580cfd90 0x7fd0580b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 --2- 172.21.15.22:0/254249827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0580709a0 0x7fd0580b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 >> 172.21.15.22:0/254249827 conn(0x7fd05809aa30 msgr2=0x7fd0580b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 shutdown_connections 2024-04-27T07:51:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.687+0000 7fd05e95d640 1 -- 172.21.15.22:0/254249827 wait complete. 2024-04-27T07:51:25.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 Processor -- start 2024-04-27T07:51:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 -- start start 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fed9406f750 0x7fed94167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed940cdb10 con 0x7fed940cffe0 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed940cddf0 con 0x7fed94071f70 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9c603640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fed940ce0d0 con 0x7fed9406f750 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fed940cffe0 0x7fed94164390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54196/0 (socket says 172.21.15.22:54196) 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 -- 172.21.15.22:0/2394071920 learned_addr learned my addr 172.21.15.22:0/2394071920 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9aba0640 1 --2- 172.21.15.22:0/2394071920 >> v2:172.21.15.183:3300/0 conn(0x7fed9406f750 0x7fed94167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265280609 0 0) 0x7fed940cdb10 con 0x7fed940cffe0 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed80003530 con 0x7fed940cffe0 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128692474 0 0) 0x7fed80003530 con 0x7fed940cffe0 2024-04-27T07:51:25.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 >> v2:172.21.15.183:3300/0 conn(0x7fed9406f750 msgr2=0x7fed94167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 --2- 172.21.15.22:0/2394071920 >> v2:172.21.15.183:3300/0 conn(0x7fed9406f750 0x7fed94167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 >> v1:172.21.15.183:6789/0 conn(0x7fed94071f70 legacy=0x7fed94072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9406e220 con 0x7fed940cffe0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fed84001680 con 0x7fed940cffe0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 >> v1:172.21.15.22:6789/0 conn(0x7fed940cffe0 legacy=0x7fed94164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed800039c0 0x7fed80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 0x7fed80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed800051d0 0x7fed8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed940ce0d0 con 0x7fed80004810 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed8000c170 con 0x7fed800051d0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fed8000c370 con 0x7fed800039c0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 0x7fed80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9aba0640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed800051d0 0x7fed8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 -- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed800039c0 msgr2=0x7fed80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed800039c0 0x7fed80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 -- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed800051d0 msgr2=0x7fed8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed800051d0 0x7fed8000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed9aba0640 1 -- 172.21.15.22:0/2394071920 reap_dead start 2024-04-27T07:51:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.831+0000 7fed99b9e640 1 -- 172.21.15.22:0/2394071920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed80003530 con 0x7fed80004810 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed99b9e640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 0x7fed80004c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c210 tx=0x7fed8401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4417a8da4a3fa975 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84007d40 con 0x7fed80004810 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed840a9070 con 0x7fed80004810 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9939d640 1 -- 172.21.15.22:0/2394071920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84005d80 con 0x7fed80004810 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 msgr2=0x7fed80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 0x7fed80004c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c210 tx=0x7fed8401cc60 comp rx=0 tx=0).stop 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- 172.21.15.22:0/2394071920 shutdown_connections 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 --2- 172.21.15.22:0/2394071920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed80004810 0x7fed80004c90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- 172.21.15.22:0/2394071920 >> 172.21.15.22:0/2394071920 conn(0x7fed940b21d0 msgr2=0x7fed940b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:25.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- 172.21.15.22:0/2394071920 shutdown_connections 2024-04-27T07:51:25.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- 172.21.15.22:0/2394071920 wait complete. 2024-04-27T07:51:25.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 Processor -- start 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- start start 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 0x7fed9413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9415e280 0x7fed9415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed9406dfd0 con 0x7fed9415bff0 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed940cd8c0 con 0x7fed9415e280 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed9c603640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fed940cceb0 con 0x7fed80004810 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed99b9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed99b9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58532/0 (socket says 172.21.15.22:58532) 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.835+0000 7fed99b9e640 1 -- 172.21.15.22:0/1962002468 learned_addr learned my addr 172.21.15.22:0/1962002468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9a39f640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 0x7fed9413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9aba0640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9415e280 0x7fed9415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 -- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 msgr2=0x7fed9413e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 0x7fed9413e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 -- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9415e280 msgr2=0x7fed9415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9415e280 0x7fed9415e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 -- 172.21.15.22:0/1962002468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed941ea620 con 0x7fed9415bff0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9a39f640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 0x7fed9413e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed99b9e640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed8401c6d0 tx=0x7fed80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84007140 con 0x7fed9415bff0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed840072e0 con 0x7fed9415bff0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed94140ac0 con 0x7fed9415bff0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84007690 con 0x7fed9415bff0 2024-04-27T07:51:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed941ea890 con 0x7fed9415bff0 2024-04-27T07:51:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9404ff10 con 0x7fed9415bff0 2024-04-27T07:51:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed840b6030 con 0x7fed9415bff0 2024-04-27T07:51:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 0x7fed7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fed8411fa40 con 0x7fed9415bff0 2024-04-27T07:51:25.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9a39f640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 0x7fed7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:25.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.839+0000 7fed9a39f640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 0x7fed7806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fed940b1670 tx=0x7fed90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:25.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:25.843+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed840169b0 con 0x7fed9415bff0 2024-04-27T07:51:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.003+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed940d12e0 con 0x7fed9415bff0 2024-04-27T07:51:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed8affd640 1 -- 172.21.15.22:0/1962002468 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fed840f0000 con 0x7fed9415bff0 2024-04-27T07:51:26.011 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 msgr2=0x7fed7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 0x7fed7806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fed940b1670 tx=0x7fed90009040 comp rx=0 tx=0).stop 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 msgr2=0x7fed941401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed8401c6d0 tx=0x7fed80005840 comp rx=0 tx=0).stop 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 shutdown_connections 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fed78068380 0x7fed7806a840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9415e280 0x7fed9415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed9415bff0 0x7fed941401c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 --2- 172.21.15.22:0/1962002468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed80004810 0x7fed9413e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 >> 172.21.15.22:0/1962002468 conn(0x7fed940b21d0 msgr2=0x7fed941459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 shutdown_connections 2024-04-27T07:51:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.007+0000 7fed9c603640 1 -- 172.21.15.22:0/1962002468 wait complete. 2024-04-27T07:51:26.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-27T07:51:26.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 Processor -- start 2024-04-27T07:51:26.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 -- start start 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09500cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09500720d0 con 0x7f09500cfc40 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0950072370 con 0x7f0950144dd0 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0956ffd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0950072650 con 0x7f095006f870 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f095559a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f09500cfc40 0x7f09500b3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54206/0 (socket says 172.21.15.22:54206) 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f095559a640 1 -- 172.21.15.22:0/4131787448 learned_addr learned my addr 172.21.15.22:0/4131787448 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:26.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09500cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256184806 0 0) 0x7f09500720d0 con 0x7f09500cfc40 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0940003530 con 0x7f09500cfc40 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1627334793 0 0) 0x7f0940003530 con 0x7f09500cfc40 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 msgr2=0x7f09500cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09500cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 >> v1:172.21.15.183:6789/0 conn(0x7f0950144dd0 legacy=0x7f095006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0950072930 con 0x7f09500cfc40 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0948001a00 con 0x7f09500cfc40 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 >> v1:172.21.15.22:6789/0 conn(0x7f09500cfc40 legacy=0x7f09500b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0940003f70 0x7f0940004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0940004d50 0x7f09400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0950072650 con 0x7f0940003f70 2024-04-27T07:51:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f094000bab0 con 0x7f0940004d50 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f094000bcb0 con 0x7f095006f870 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0940003f70 0x7f0940004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0954d99640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 -- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 msgr2=0x7f09400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 -- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0940004d50 msgr2=0x7f09400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0940004d50 0x7f09400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f094ffff640 1 -- 172.21.15.22:0/4131787448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0940003530 con 0x7f0940003f70 2024-04-27T07:51:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.151+0000 7f0954d99640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f095006f870 0x7f09400039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f094ffff640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0940003f70 0x7f0940004810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f093c009ac0 tx=0x7f093c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9b19002ea2f630 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c0b2070 con 0x7f0940003f70 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c0a5640 con 0x7f0940003f70 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f094f7fe640 1 -- 172.21.15.22:0/4131787448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c009690 con 0x7f0940003f70 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- 172.21.15.22:0/4131787448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0940003f70 msgr2=0x7f0940004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 --2- 172.21.15.22:0/4131787448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0940003f70 0x7f0940004810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f093c009ac0 tx=0x7f093c01ca50 comp rx=0 tx=0).stop 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f095559a640 1 -- 172.21.15.22:0/4131787448 reap_dead start 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- 172.21.15.22:0/4131787448 shutdown_connections 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- 172.21.15.22:0/4131787448 >> 172.21.15.22:0/4131787448 conn(0x7f0950067f10 msgr2=0x7f095006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- 172.21.15.22:0/4131787448 shutdown_connections 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- 172.21.15.22:0/4131787448 wait complete. 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 Processor -- start 2024-04-27T07:51:26.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- start start 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0940003f70 0x7f095013e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095013de00 0x7f09500b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09500d00b0 con 0x7f095013b0f0 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09501451b0 con 0x7f095013de00 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.155+0000 7f0956ffd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f09500d0230 con 0x7f0940003f70 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58560/0 (socket says 172.21.15.22:58560) 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 -- 172.21.15.22:0/1284617939 learned_addr learned my addr 172.21.15.22:0/1284617939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094ffff640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095013de00 0x7f09500b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f0954d99640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0940003f70 0x7f095013e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 -- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0940003f70 msgr2=0x7f095013e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0940003f70 0x7f095013e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 -- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095013de00 msgr2=0x7f09500b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095013de00 0x7f09500b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 -- 172.21.15.22:0/1284617939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095013e290 con 0x7f095013b0f0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f095559a640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0950070440 tx=0x7f094800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094801b070 con 0x7f095013b0f0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094800da20 con 0x7f095013b0f0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09501f57b0 con 0x7f095013b0f0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094800de10 con 0x7f095013b0f0 2024-04-27T07:51:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501f5d70 con 0x7f095013b0f0 2024-04-27T07:51:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0950068c30 con 0x7f095013b0f0 2024-04-27T07:51:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f094801e070 con 0x7f095013b0f0 2024-04-27T07:51:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0948089040 con 0x7f095013b0f0 2024-04-27T07:51:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.159+0000 7f0954d99640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.163+0000 7f0954d99640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 0x7f093406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f095015b520 tx=0x7f0944009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.163+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0948059600 con 0x7f095013b0f0 2024-04-27T07:51:26.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:26.387+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7f095006bee0 con 0x7f095013b0f0 2024-04-27T07:51:27.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:27.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:27.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.331+0000 7f094d7fa640 1 -- 172.21.15.22:0/1284617939 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v51) ==== 123+0+0 (secure 0 0 0) 0x7f0948058f00 con 0x7f095013b0f0 2024-04-27T07:51:27.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 msgr2=0x7f093406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 0x7f093406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f095015b520 tx=0x7f0944009040 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 msgr2=0x7f09500b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0950070440 tx=0x7f094800fcd0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 shutdown_connections 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f095013b0f0 0x7f09500b2a80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095013de00 0x7f09500b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 --2- 172.21.15.22:0/1284617939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0940003f70 0x7f095013e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 >> 172.21.15.22:0/1284617939 conn(0x7f0950067f10 msgr2=0x7f095015d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 shutdown_connections 2024-04-27T07:51:27.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.335+0000 7f0956ffd640 1 -- 172.21.15.22:0/1284617939 wait complete. 2024-04-27T07:51:27.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-27T07:51:27.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 Processor -- start 2024-04-27T07:51:27.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 -- start start 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8a3c0cbb10 0x7f8a3c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c0cc4e0 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c070260 con 0x7f8a3c0ccf90 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a43ab7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3c06c3b0 con 0x7f8a3c0cbb10 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8a3c0cd6e0 0x7f8a3c06fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54216/0 (socket says 172.21.15.22:54216) 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 -- 172.21.15.22:0/3458485756 learned_addr learned my addr 172.21.15.22:0/3458485756 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a42054640 1 --2- 172.21.15.22:0/3458485756 >> v2:172.21.15.183:3300/0 conn(0x7f8a3c0cbb10 0x7f8a3c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493378589 0 0) 0x7f8a3c0cc4e0 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a28003530 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 514424292 0 0) 0x7f8a28003530 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 >> v2:172.21.15.183:3300/0 conn(0x7f8a3c0cbb10 msgr2=0x7f8a3c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 --2- 172.21.15.22:0/3458485756 >> v2:172.21.15.183:3300/0 conn(0x7f8a3c0cbb10 0x7f8a3c0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 >> v1:172.21.15.183:6789/0 conn(0x7f8a3c0ccf90 legacy=0x7f8a3c06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c06c690 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f8a2c001a00 con 0x7f8a3c0cd6e0 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 >> v1:172.21.15.22:6789/0 conn(0x7f8a3c0cd6e0 legacy=0x7f8a3c06fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a280039c0 0x7f8a28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 0x7f8a28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a280051d0 0x7f8a2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c06c3b0 con 0x7f8a28004810 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2800c070 con 0x7f8a280039c0 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2800c270 con 0x7f8a280051d0 2024-04-27T07:51:27.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 0x7f8a28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 -- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a280051d0 msgr2=0x7f8a2800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a280051d0 0x7f8a2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41853640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a280039c0 0x7f8a28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 -- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a280039c0 msgr2=0x7f8a28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a280039c0 0x7f8a28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a42054640 1 -- 172.21.15.22:0/3458485756 reap_dead start 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41052640 1 -- 172.21.15.22:0/3458485756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a28003530 con 0x7f8a28004810 2024-04-27T07:51:27.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.487+0000 7f8a41853640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a280039c0 0x7f8a28003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:27.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a41052640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 0x7f8a28004c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a850 tx=0x7f8a2c01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58131059370c375 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:27.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00a4a0 con 0x7f8a28004810 2024-04-27T07:51:27.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0aa710 con 0x7f8a28004810 2024-04-27T07:51:27.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a40851640 1 -- 172.21.15.22:0/3458485756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0aab40 con 0x7f8a28004810 2024-04-27T07:51:27.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 -- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 msgr2=0x7f8a28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 0x7f8a28004c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a850 tx=0x7f8a2c01fac0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 -- 172.21.15.22:0/3458485756 shutdown_connections 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/3458485756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a28004810 0x7f8a28004c90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 -- 172.21.15.22:0/3458485756 >> 172.21.15.22:0/3458485756 conn(0x7f8a3c09aa30 msgr2=0x7f8a3c0b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 -- 172.21.15.22:0/3458485756 shutdown_connections 2024-04-27T07:51:27.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.491+0000 7f8a43ab7640 1 -- 172.21.15.22:0/3458485756 wait complete. 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 Processor -- start 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- start start 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 0x7f8a3c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a3c0b4f60 0x7f8a3c151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0cbfa0 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c06f630 con 0x7f8a28004810 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c0cc120 con 0x7f8a3c0b4f60 2024-04-27T07:51:27.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58576/0 (socket says 172.21.15.22:58576) 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 -- 172.21.15.22:0/1599820761 learned_addr learned my addr 172.21.15.22:0/1599820761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41853640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 0x7f8a3c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a42054640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a3c0b4f60 0x7f8a3c151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 -- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a3c0b4f60 msgr2=0x7f8a3c151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a3c0b4f60 0x7f8a3c151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 -- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 msgr2=0x7f8a3c0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 0x7f8a3c0b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 -- 172.21.15.22:0/1599820761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c0b5530 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41853640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 0x7f8a3c0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a41052640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c01f8f0 tx=0x7f8a28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00a480 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0aa8c0 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c0b5810 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0aac70 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c0cacd0 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c04ff10 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.495+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a2c0c4060 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.499+0000 7f8a327fc640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 0x7f8a2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:27.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.499+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8a2c124600 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.499+0000 7f8a41853640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 0x7f8a2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:27.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.499+0000 7f8a41853640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 0x7f8a2006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c10 tx=0x7f8a38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:27.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.503+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a2c0ae030 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.663+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8a3c0b59e0 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.667+0000 7f8a327fc640 1 -- 172.21.15.22:0/1599820761 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f8a2c0197d0 con 0x7f8a3c0b4ae0 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 msgr2=0x7f8a2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 0x7f8a2006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c10 tx=0x7f8a38009040 comp rx=0 tx=0).stop 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 msgr2=0x7f8a3c1514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c01f8f0 tx=0x7f8a28005840 comp rx=0 tx=0).stop 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 shutdown_connections 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8a20068380 0x7f8a2006a840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a3c0b4f60 0x7f8a3c151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a3c0b4ae0 0x7f8a3c1514c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 --2- 172.21.15.22:0/1599820761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a28004810 0x7f8a3c0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 >> 172.21.15.22:0/1599820761 conn(0x7f8a3c09aa30 msgr2=0x7f8a3c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 shutdown_connections 2024-04-27T07:51:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:27.671+0000 7f8a43ab7640 1 -- 172.21.15.22:0/1599820761 wait complete. 2024-04-27T07:51:27.693 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-27T07:51:27.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:51:27.744 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:51:27.794 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-27T07:51:27.795 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:27.795 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:27.801 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:27.801 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:27.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:28.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.015+0000 7fa1afc6a640 1 Processor -- start 2024-04-27T07:51:28.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.015+0000 7fa1afc6a640 1 -- start start 2024-04-27T07:51:28.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.015+0000 7fa1afc6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa1a80728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1afc6a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1a8072e00 0x7fa1a806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1afc6a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a81520d0 con 0x7fa1a8151cf0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1afc6a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a8073250 con 0x7fa1a8072e00 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1afc6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a80b3590 con 0x7fa1a80cea10 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa1a8151cf0 0x7fa1a80b5200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54226/0 (socket says 172.21.15.22:54226) 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 -- 172.21.15.22:0/2137712731 learned_addr learned my addr 172.21.15.22:0/2137712731 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ad205640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa1a80728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ae207640 1 --2- 172.21.15.22:0/2137712731 >> v2:172.21.15.183:3300/0 conn(0x7fa1a8072e00 0x7fa1a806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714387255 0 0) 0x7fa1a81520d0 con 0x7fa1a8151cf0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa194003530 con 0x7fa1a8151cf0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2614050094 0 0) 0x7fa194003530 con 0x7fa1a8151cf0 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 msgr2=0x7fa1a80728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa1a80728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 >> v2:172.21.15.183:3300/0 conn(0x7fa1a8072e00 msgr2=0x7fa1a806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 --2- 172.21.15.22:0/2137712731 >> v2:172.21.15.183:3300/0 conn(0x7fa1a8072e00 0x7fa1a806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a80b3870 con 0x7fa1a8151cf0 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fa1a4001680 con 0x7fa1a8151cf0 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 >> v1:172.21.15.22:6789/0 conn(0x7fa1a8151cf0 legacy=0x7fa1a80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1940039c0 0x7fa194003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa194004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194004d50 0x7fa1940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8073250 con 0x7fa1940039c0 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80b3590 con 0x7fa194004d50 2024-04-27T07:51:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa19400ba00 con 0x7fa1a80cea10 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1940039c0 0x7fa194003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ae207640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194004d50 0x7fa1940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ad205640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa194004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 -- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 msgr2=0x7fa194004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a80cea10 0x7fa194004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 -- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194004d50 msgr2=0x7fa1940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194004d50 0x7fa1940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 -- 172.21.15.22:0/2137712731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa194003530 con 0x7fa1940039c0 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1ada06640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1940039c0 0x7fa194003de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c240 tx=0x7fa1a401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d680b6635e8d0fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007d40 con 0x7fa1940039c0 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40a9020 con 0x7fa1940039c0 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.019+0000 7fa1aca04640 1 -- 172.21.15.22:0/2137712731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40a5b70 con 0x7fa1940039c0 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/2137712731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1940039c0 msgr2=0x7fa194003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/2137712731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1940039c0 0x7fa194003de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c240 tx=0x7fa1a401cc70 comp rx=0 tx=0).stop 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ae207640 1 -- 172.21.15.22:0/2137712731 reap_dead start 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/2137712731 shutdown_connections 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/2137712731 >> 172.21.15.22:0/2137712731 conn(0x7fa1a8067f10 msgr2=0x7fa1a806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/2137712731 shutdown_connections 2024-04-27T07:51:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/2137712731 wait complete. 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 Processor -- start 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- start start 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1940039c0 0x7fa1a81f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a8154f20 0x7fa1a815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a806f910 con 0x7fa1a8154aa0 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80b5910 con 0x7fa1940039c0 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8072030 con 0x7fa1a8154f20 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58604/0 (socket says 172.21.15.22:58604) 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 -- 172.21.15.22:0/669212123 learned_addr learned my addr 172.21.15.22:0/669212123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ada06640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1940039c0 0x7fa1a81f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ae207640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a8154f20 0x7fa1a815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 -- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a8154f20 msgr2=0x7fa1a815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a8154f20 0x7fa1a815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 -- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1940039c0 msgr2=0x7fa1a81f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1940039c0 0x7fa1a81f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 -- 172.21.15.22:0/669212123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8154740 con 0x7fa1a8154aa0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1ad205640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1980115c0 tx=0x7fa198011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198004220 con 0x7fa1a8154aa0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1980043c0 con 0x7fa1a8154aa0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8153560 con 0x7fa1a8154aa0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19800f8f0 con 0x7fa1a8154aa0 2024-04-27T07:51:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.023+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a8153b20 con 0x7fa1a8154aa0 2024-04-27T07:51:28.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8068c30 con 0x7fa1a8154aa0 2024-04-27T07:51:28.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa19800fa90 con 0x7fa1a8154aa0 2024-04-27T07:51:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa19e7fc640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 0x7fa18c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa198086fe0 con 0x7fa1a8154aa0 2024-04-27T07:51:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa1ada06640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 0x7fa18c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.027+0000 7fa1ada06640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 0x7fa18c06a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40aa000 tx=0x7fa1a81557f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.031+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa198019030 con 0x7fa1a8154aa0 2024-04-27T07:51:28.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.195+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1a806bf60 con 0x7fa1a8154aa0 2024-04-27T07:51:28.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa19e7fc640 1 -- 172.21.15.22:0/669212123 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fa198057590 con 0x7fa1a8154aa0 2024-04-27T07:51:28.203 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:28.203 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:28.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 msgr2=0x7fa18c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 0x7fa18c06a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40aa000 tx=0x7fa1a81557f0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 msgr2=0x7fa1a815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1980115c0 tx=0x7fa198011a80 comp rx=0 tx=0).stop 2024-04-27T07:51:28.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 shutdown_connections 2024-04-27T07:51:28.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa18c0683b0 0x7fa18c06a870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1a8154f20 0x7fa1a815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a8154aa0 0x7fa1a815a2b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 --2- 172.21.15.22:0/669212123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1940039c0 0x7fa1a81f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 >> 172.21.15.22:0/669212123 conn(0x7fa1a8067f10 msgr2=0x7fa1a8150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 shutdown_connections 2024-04-27T07:51:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.199+0000 7fa1afc6a640 1 -- 172.21.15.22:0/669212123 wait complete. 2024-04-27T07:51:28.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:28.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:28.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 Processor -- start 2024-04-27T07:51:28.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 -- start start 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd1f8071f70 0x7fd1f8072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f8072890 con 0x7fd1f80cffe0 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd1f80ce0d0 con 0x7fd1f8071f70 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1fdf48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f80cd8c0 con 0x7fd1f806ee50 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f77fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd1f8071f70 0x7fd1f8072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f7fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f6ffd640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fd1f80cffe0 0x7fd1f806e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:40662/0 (socket says 172.21.15.22:40662) 2024-04-27T07:51:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f6ffd640 1 -- 172.21.15.22:0/1749314451 learned_addr learned my addr 172.21.15.22:0/1749314451 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:28.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f7fff640 1 -- 172.21.15.22:0/1749314451 >> v2:172.21.15.183:3300/0 conn(0x7fd1f8071f70 msgr2=0x7fd1f8072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f7fff640 1 --2- 172.21.15.22:0/1749314451 >> v2:172.21.15.183:3300/0 conn(0x7fd1f8071f70 0x7fd1f8072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f7fff640 1 -- 172.21.15.22:0/1749314451 >> v1:172.21.15.183:6789/0 conn(0x7fd1f80cffe0 legacy=0x7fd1f806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.343+0000 7fd1f7fff640 1 -- 172.21.15.22:0/1749314451 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f80cdb10 con 0x7fd1f806ee50 2024-04-27T07:51:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1f7fff640 1 --2- 172.21.15.22:0/1749314451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f806f2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1fbba6243281253 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1f67fc640 1 -- 172.21.15.22:0/1749314451 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00b2070 con 0x7fd1f806ee50 2024-04-27T07:51:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1f67fc640 1 -- 172.21.15.22:0/1749314451 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f00a57a0 con 0x7fd1f806ee50 2024-04-27T07:51:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1f67fc640 1 -- 172.21.15.22:0/1749314451 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00a5b70 con 0x7fd1f806ee50 2024-04-27T07:51:28.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- 172.21.15.22:0/1749314451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 msgr2=0x7fd1f806f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/1749314451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f806f2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f001c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- 172.21.15.22:0/1749314451 shutdown_connections 2024-04-27T07:51:28.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/1749314451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f806f2c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/1749314451 >> v2:172.21.15.183:3300/0 conn(0x7fd1f8071f70 0x7fd1f8072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- 172.21.15.22:0/1749314451 >> 172.21.15.22:0/1749314451 conn(0x7fd1f80b21d0 msgr2=0x7fd1f80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- 172.21.15.22:0/1749314451 shutdown_connections 2024-04-27T07:51:28.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- 172.21.15.22:0/1749314451 wait complete. 2024-04-27T07:51:28.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 Processor -- start 2024-04-27T07:51:28.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.347+0000 7fd1fdf48640 1 -- start start 2024-04-27T07:51:28.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f815f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f80cffe0 0x7fd1f815e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8073130 con 0x7fd1f8071f70 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8072fb0 con 0x7fd1f80cffe0 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f80cceb0 con 0x7fd1f806ee50 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58620/0 (socket says 172.21.15.22:58620) 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 -- 172.21.15.22:0/3939882452 learned_addr learned my addr 172.21.15.22:0/3939882452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f7fff640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f80cffe0 0x7fd1f815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f77fe640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 -- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 msgr2=0x7fd1f815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 -- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f80cffe0 msgr2=0x7fd1f815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f80cffe0 0x7fd1f815e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 -- 172.21.15.22:0/3939882452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f815d010 con 0x7fd1f8071f70 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f6ffd640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c370 tx=0x7fd1e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0017070 con 0x7fd1f8071f70 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e0005cf0 con 0x7fd1f8071f70 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f815d2b0 con 0x7fd1f8071f70 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00072f0 con 0x7fd1f8071f70 2024-04-27T07:51:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f815b340 con 0x7fd1f8071f70 2024-04-27T07:51:28.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f804ff10 con 0x7fd1f8071f70 2024-04-27T07:51:28.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd1e001a070 con 0x7fd1f8071f70 2024-04-27T07:51:28.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 0x7fd1d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd1e0085c80 con 0x7fd1f8071f70 2024-04-27T07:51:28.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.351+0000 7fd1f77fe640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 0x7fd1d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.355+0000 7fd1f77fe640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 0x7fd1d806a870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8143270 tx=0x7fd1e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.355+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd1e00562d0 con 0x7fd1f8071f70 2024-04-27T07:51:28.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.519+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1f80d11e0 con 0x7fd1f8071f70 2024-04-27T07:51:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.519+0000 7fd1effff640 1 -- 172.21.15.22:0/3939882452 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd1e0055bd0 con 0x7fd1f8071f70 2024-04-27T07:51:28.526 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 msgr2=0x7fd1d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 0x7fd1d806a870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8143270 tx=0x7fd1e8008040 comp rx=0 tx=0).stop 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 msgr2=0x7fd1f815ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c370 tx=0x7fd1e000c830 comp rx=0 tx=0).stop 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 shutdown_connections 2024-04-27T07:51:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd1d80683b0 0x7fd1d806a870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f80cffe0 0x7fd1f815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1f8071f70 0x7fd1f815ae00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 --2- 172.21.15.22:0/3939882452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1f806ee50 0x7fd1f815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 >> 172.21.15.22:0/3939882452 conn(0x7fd1f80b21d0 msgr2=0x7fd1f8143b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 shutdown_connections 2024-04-27T07:51:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.523+0000 7fd1fdf48640 1 -- 172.21.15.22:0/3939882452 wait complete. 2024-04-27T07:51:28.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:28.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 Processor -- start 2024-04-27T07:51:28.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 -- start start 2024-04-27T07:51:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f01980d1070 0x7f01980b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f01980b2c60 0x7f019806bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f019806c110 0x7f01981626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f019806c560 con 0x7f01980d1070 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f019815f550 con 0x7f01980b2c60 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019ff5a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f019815f830 con 0x7f019806c110 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f01980d1070 0x7f01980b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f01980d1070 0x7f01980b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58636/0 (socket says 172.21.15.22:58636) 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 -- 172.21.15.22:0/2314543661 learned_addr learned my addr 172.21.15.22:0/2314543661 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019d4f5640 1 --2- 172.21.15.22:0/2314543661 >> v2:172.21.15.183:3300/0 conn(0x7f01980b2c60 0x7f019806bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019e4f7640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f019806c110 0x7f01981626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 -- 172.21.15.22:0/2314543661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f019806c110 msgr2=0x7f01981626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f019806c110 0x7f01981626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 -- 172.21.15.22:0/2314543661 >> v2:172.21.15.183:3300/0 conn(0x7f01980b2c60 msgr2=0x7f019806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 --2- 172.21.15.22:0/2314543661 >> v2:172.21.15.183:3300/0 conn(0x7f01980b2c60 0x7f019806bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.663+0000 7f019dcf6640 1 -- 172.21.15.22:0/2314543661 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019815fb10 con 0x7f01980d1070 2024-04-27T07:51:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019dcf6640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01980d1070 0x7f01980b2720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0198071a20 tx=0x7f019401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c7628fbf138a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ccf4640 1 -- 172.21.15.22:0/2314543661 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940a6500 con 0x7f01980d1070 2024-04-27T07:51:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ccf4640 1 -- 172.21.15.22:0/2314543661 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01940a66a0 con 0x7f01980d1070 2024-04-27T07:51:28.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ccf4640 1 -- 172.21.15.22:0/2314543661 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940a6a70 con 0x7f01980d1070 2024-04-27T07:51:28.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 -- 172.21.15.22:0/2314543661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01980d1070 msgr2=0x7f01980b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01980d1070 0x7f01980b2720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0198071a20 tx=0x7f019401c670 comp rx=0 tx=0).stop 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 -- 172.21.15.22:0/2314543661 shutdown_connections 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f019806c110 0x7f01981626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 --2- 172.21.15.22:0/2314543661 >> v2:172.21.15.183:3300/0 conn(0x7f01980b2c60 0x7f019806bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 --2- 172.21.15.22:0/2314543661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01980d1070 0x7f01980b2720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 -- 172.21.15.22:0/2314543661 >> 172.21.15.22:0/2314543661 conn(0x7f019809aa30 msgr2=0x7f01980b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 -- 172.21.15.22:0/2314543661 shutdown_connections 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.667+0000 7f019ff5a640 1 -- 172.21.15.22:0/2314543661 wait complete. 2024-04-27T07:51:28.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 Processor -- start 2024-04-27T07:51:28.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- start start 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01980b2c60 0x7f01981fb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 0x7f01981fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f019815e560 con 0x7f019806c110 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f019815e3e0 con 0x7f01980d1070 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f019815e6e0 con 0x7f01980b2c60 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58650/0 (socket says 172.21.15.22:58650) 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 -- 172.21.15.22:0/3031060766 learned_addr learned my addr 172.21.15.22:0/3031060766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019d4f5640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01980b2c60 0x7f01981fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019e4f7640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 0x7f01981fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 -- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01980b2c60 msgr2=0x7f01981fb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01980b2c60 0x7f01981fb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 -- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 msgr2=0x7f01981fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 0x7f01981fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 -- 172.21.15.22:0/3031060766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01980aefc0 con 0x7f019806c110 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019e4f7640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 0x7f01981fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019dcf6640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f019401ad20 tx=0x7f0194019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940a6c60 con 0x7f019806c110 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01940a6e00 con 0x7f019806c110 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0198160ba0 con 0x7f019806c110 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940af6c0 con 0x7f019806c110 2024-04-27T07:51:28.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0198161160 con 0x7f019806c110 2024-04-27T07:51:28.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019804ff10 con 0x7f019806c110 2024-04-27T07:51:28.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01940af860 con 0x7f019806c110 2024-04-27T07:51:28.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 0x7f017c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.671+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0194120f60 con 0x7f019806c110 2024-04-27T07:51:28.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.675+0000 7f019d4f5640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 0x7f017c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.675+0000 7f019d4f5640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 0x7f017c06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f019815a820 tx=0x7f0188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.679+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01940165e0 con 0x7f019806c110 2024-04-27T07:51:28.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.839+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f019815f1e0 con 0x7f019806c110 2024-04-27T07:51:28.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f018e7fc640 1 -- 172.21.15.22:0/3031060766 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f01940f1520 con 0x7f019806c110 2024-04-27T07:51:28.847 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 msgr2=0x7f017c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 0x7f017c06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f019815a820 tx=0x7f0188008040 comp rx=0 tx=0).stop 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 msgr2=0x7f01980afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f019401ad20 tx=0x7f0194019410 comp rx=0 tx=0).stop 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 shutdown_connections 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f017c068380 0x7f017c06a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01980d1070 0x7f01981fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01980b2c60 0x7f01981fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 --2- 172.21.15.22:0/3031060766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f019806c110 0x7f01980afbe0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 >> 172.21.15.22:0/3031060766 conn(0x7f019809aa30 msgr2=0x7f01980cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 shutdown_connections 2024-04-27T07:51:28.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.843+0000 7f019ff5a640 1 -- 172.21.15.22:0/3031060766 wait complete. 2024-04-27T07:51:28.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:28.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 Processor -- start 2024-04-27T07:51:28.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 -- start start 2024-04-27T07:51:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0b00cffe0 0x7fd0b0164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd0b006f750 0x7fd0b01668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b00cdb10 con 0x7fd0b0071f70 2024-04-27T07:51:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b00ce0d0 con 0x7fd0b00cffe0 2024-04-27T07:51:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b7d99640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b00cd8c0 con 0x7fd0b006f750 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5334640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0b00cffe0 0x7fd0b0164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5b35640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd0b0071f70 0x7fd0b0072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54238/0 (socket says 172.21.15.22:54238) 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5b35640 1 -- 172.21.15.22:0/3525235782 learned_addr learned my addr 172.21.15.22:0/3525235782 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5334640 1 -- 172.21.15.22:0/3525235782 >> v2:172.21.15.183:3300/0 conn(0x7fd0b006f750 msgr2=0x7fd0b01668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5334640 1 --2- 172.21.15.22:0/3525235782 >> v2:172.21.15.183:3300/0 conn(0x7fd0b006f750 0x7fd0b01668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5334640 1 -- 172.21.15.22:0/3525235782 >> v1:172.21.15.22:6789/0 conn(0x7fd0b0071f70 legacy=0x7fd0b0072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.979+0000 7fd0b5334640 1 -- 172.21.15.22:0/3525235782 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b00cddf0 con 0x7fd0b00cffe0 2024-04-27T07:51:28.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- 172.21.15.22:0/3525235782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0164390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0071f00 tx=0x7fd0a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852556fc98673fb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b4b33640 1 -- 172.21.15.22:0/3525235782 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00b0070 con 0x7fd0b00cffe0 2024-04-27T07:51:28.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b4b33640 1 -- 172.21.15.22:0/3525235782 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a00ab600 con 0x7fd0b00cffe0 2024-04-27T07:51:28.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b4b33640 1 -- 172.21.15.22:0/3525235782 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00ab9d0 con 0x7fd0b00cffe0 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3525235782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b00cffe0 msgr2=0x7fd0b0164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3525235782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0164390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0071f00 tx=0x7fd0a001c3f0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3525235782 shutdown_connections 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3525235782 >> v2:172.21.15.183:3300/0 conn(0x7fd0b006f750 0x7fd0b01668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3525235782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0164390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3525235782 >> 172.21.15.22:0/3525235782 conn(0x7fd0b00b21d0 msgr2=0x7fd0b00b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3525235782 shutdown_connections 2024-04-27T07:51:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3525235782 wait complete. 2024-04-27T07:51:28.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 Processor -- start 2024-04-27T07:51:28.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- start start 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0b006f750 0x7fd0b015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b00cd030 con 0x7fd0b0071f70 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b00cceb0 con 0x7fd0b00cffe0 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b0169640 con 0x7fd0b006f750 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58654/0 (socket says 172.21.15.22:58654) 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 -- 172.21.15.22:0/3446182874 learned_addr learned my addr 172.21.15.22:0/3446182874 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5b35640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0b006f750 0x7fd0b015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 -- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0b006f750 msgr2=0x7fd0b015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0b006f750 0x7fd0b015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 -- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0b00cffe0 msgr2=0x7fd0b0142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 -- 172.21.15.22:0/3446182874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b015fb80 con 0x7fd0b0071f70 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b5334640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ae040 tx=0x7fd0a0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00b8020 con 0x7fd0b0071f70 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a00b4610 con 0x7fd0b0071f70 2024-04-27T07:51:28.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00b0040 con 0x7fd0b0071f70 2024-04-27T07:51:28.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.983+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b013cbe0 con 0x7fd0b0071f70 2024-04-27T07:51:28.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b013c100 con 0x7fd0b0071f70 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b004ff10 con 0x7fd0b0071f70 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0a00bb020 con 0x7fd0b0071f70 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0a67fc640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd0a01206f0 con 0x7fd0b0071f70 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0b5b35640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.987+0000 7fd0b5b35640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 0x7fd09406a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac000f70 tx=0x7fd0ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:28.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:28.991+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0a00c2030 con 0x7fd0b0071f70 2024-04-27T07:51:29.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.151+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd0b00723f0 con 0x7fd0b0071f70 2024-04-27T07:51:29.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.151+0000 7fd0a67fc640 1 -- 172.21.15.22:0/3446182874 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fd0b015fb80 con 0x7fd0b0071f70 2024-04-27T07:51:29.158 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 msgr2=0x7fd09406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 0x7fd09406a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac000f70 tx=0x7fd0ac009040 comp rx=0 tx=0).stop 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 msgr2=0x7fd0b01427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ae040 tx=0x7fd0a0019190 comp rx=0 tx=0).stop 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 shutdown_connections 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0b00cffe0 0x7fd0b0142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0b0071f70 0x7fd0b01427f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 --2- 172.21.15.22:0/3446182874 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0b006f750 0x7fd0b015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 >> 172.21.15.22:0/3446182874 conn(0x7fd0b00b21d0 msgr2=0x7fd0b0143a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:29.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 shutdown_connections 2024-04-27T07:51:29.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.155+0000 7fd0b7d99640 1 -- 172.21.15.22:0/3446182874 wait complete. 2024-04-27T07:51:29.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:29.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 Processor -- start 2024-04-27T07:51:29.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 -- start start 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 0x7f55881660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880720d0 con 0x7f5588144dd0 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5588072370 con 0x7f558806f870 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558fc5c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5588072650 con 0x7f55880cfe10 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558d9f8640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5588144dd0 0x7f558806f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54248/0 (socket says 172.21.15.22:54248) 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558d9f8640 1 -- 172.21.15.22:0/2518126388 learned_addr learned my addr 172.21.15.22:0/2518126388 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558e1f9640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 0x7f55881660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116604738 0 0) 0x7f55880720d0 con 0x7f5588144dd0 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5574003530 con 0x7f5588144dd0 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930329153 0 0) 0x7f5574003530 con 0x7f5588144dd0 2024-04-27T07:51:29.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 msgr2=0x7f55881660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 0x7f55881660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 >> v1:172.21.15.183:6789/0 conn(0x7f558806f870 legacy=0x7f55880cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.295+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588072930 con 0x7f5588144dd0 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f55840016a0 con 0x7f5588144dd0 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 >> v1:172.21.15.22:6789/0 conn(0x7f5588144dd0 legacy=0x7f558806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55740039f0 0x7f5574003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 0x7f5574004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5574004d50 0x7f55740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5588072650 con 0x7f5574004d50 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f557400ba80 con 0x7f55740039f0 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f557400bc80 con 0x7f55880cfe10 2024-04-27T07:51:29.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5574004d50 0x7f55740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558d9f8640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55740039f0 0x7f5574003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 -- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 msgr2=0x7f5574004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880cfe10 0x7f5574004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 -- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55740039f0 msgr2=0x7f5574003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55740039f0 0x7f5574003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 -- 172.21.15.22:0/2518126388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5574003530 con 0x7f5574004d50 2024-04-27T07:51:29.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558d9f8640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55740039f0 0x7f5574003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:29.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5574004d50 0x7f55740051d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5580009bd0 tx=0x7f558001eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dea90306d9848ec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55800b4070 con 0x7f5574004d50 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55800af440 con 0x7f5574004d50 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558c9f6640 1 -- 172.21.15.22:0/2518126388 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580009c10 con 0x7f5574004d50 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 -- 172.21.15.22:0/2518126388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5574004d50 msgr2=0x7f55740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 --2- 172.21.15.22:0/2518126388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5574004d50 0x7f55740051d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5580009bd0 tx=0x7f558001eeb0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558e1f9640 1 -- 172.21.15.22:0/2518126388 reap_dead start 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 -- 172.21.15.22:0/2518126388 shutdown_connections 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 -- 172.21.15.22:0/2518126388 >> 172.21.15.22:0/2518126388 conn(0x7f5588067f10 msgr2=0x7f558806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 -- 172.21.15.22:0/2518126388 shutdown_connections 2024-04-27T07:51:29.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.299+0000 7f558fc5c640 1 -- 172.21.15.22:0/2518126388 wait complete. 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 Processor -- start 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- start start 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880b2630 0x7f55881e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55881f8260 0x7f5588143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55880b4de0 con 0x7f55881f7de0 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55881451b0 con 0x7f55881f8260 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f55880b4f60 con 0x7f55880b2630 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58672/0 (socket says 172.21.15.22:58672) 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 -- 172.21.15.22:0/1712319586 learned_addr learned my addr 172.21.15.22:0/1712319586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d9f8640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880b2630 0x7f55881e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558e1f9640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55881f8260 0x7f5588143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 -- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880b2630 msgr2=0x7f55881e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880b2630 0x7f55881e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 -- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55881f8260 msgr2=0x7f5588143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55881f8260 0x7f5588143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 -- 172.21.15.22:0/1712319586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55881e7410 con 0x7f55881f7de0 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558d1f7640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f557800c560 tx=0x7f557800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780052d0 con 0x7f55881f7de0 2024-04-27T07:51:29.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780054b0 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55881f87a0 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578012870 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881e76b0 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55881f8cc0 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5578012a10 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.303+0000 7f557e7fc640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 0x7f556c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.307+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5578086570 con 0x7f55881f7de0 2024-04-27T07:51:29.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.307+0000 7f558d9f8640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 0x7f556c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.307+0000 7f558d9f8640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 0x7f556c06a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5584001ec0 tx=0x7f5584009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.311+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55881f8cc0 con 0x7f55881f7de0 2024-04-27T07:51:29.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.479+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5588068c30 con 0x7f55881f7de0 2024-04-27T07:51:29.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.479+0000 7f557e7fc640 1 -- 172.21.15.22:0/1712319586 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f5578056b30 con 0x7f55881f7de0 2024-04-27T07:51:29.485 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:29.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 msgr2=0x7f556c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 0x7f556c06a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5584001ec0 tx=0x7f5584009040 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 msgr2=0x7f5588142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f557800c560 tx=0x7f557800ca20 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 shutdown_connections 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f556c068380 0x7f556c06a840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55881f8260 0x7f5588143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55881f7de0 0x7f5588142ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 --2- 172.21.15.22:0/1712319586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55880b2630 0x7f55881e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 >> 172.21.15.22:0/1712319586 conn(0x7f5588067f10 msgr2=0x7f558815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 shutdown_connections 2024-04-27T07:51:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.483+0000 7f558fc5c640 1 -- 172.21.15.22:0/1712319586 wait complete. 2024-04-27T07:51:29.503 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:29.503 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:29.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:29.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.663+0000 7f6aa9e49640 1 Processor -- start 2024-04-27T07:51:29.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.663+0000 7f6aa9e49640 1 -- start start 2024-04-27T07:51:29.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa9e49640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6aa40d4790 0x7f6aa40d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa9e49640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa40cff70 con 0x7f6aa40b7930 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa9e49640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa40b3d30 con 0x7f6aa415ebe0 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa9e49640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa40d4540 con 0x7f6aa40d4790 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6aa40b7930 0x7f6aa40d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54254/0 (socket says 172.21.15.22:54254) 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 -- 172.21.15.22:0/45063858 learned_addr learned my addr 172.21.15.22:0/45063858 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa37fe640 1 --2- 172.21.15.22:0/45063858 >> v2:172.21.15.183:3300/0 conn(0x7f6aa40d4790 0x7f6aa40d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280146156 0 0) 0x7f6aa40cff70 con 0x7f6aa40b7930 2024-04-27T07:51:29.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a8c003530 con 0x7f6aa40b7930 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3160877693 0 0) 0x7f6a8c003530 con 0x7f6aa40b7930 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 >> v2:172.21.15.183:3300/0 conn(0x7f6aa40d4790 msgr2=0x7f6aa40d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 --2- 172.21.15.22:0/45063858 >> v2:172.21.15.183:3300/0 conn(0x7f6aa40d4790 0x7f6aa40d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 >> v1:172.21.15.183:6789/0 conn(0x7f6aa415ebe0 legacy=0x7f6aa40cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa40ba7a0 con 0x7f6aa40b7930 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f6a9c001680 con 0x7f6aa40b7930 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 >> v1:172.21.15.22:6789/0 conn(0x7f6aa40b7930 legacy=0x7f6aa40d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c0039c0 0x7f6a8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c004810 0x7f6a8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 0x7f6a8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40d4540 con 0x7f6a8c0051d0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c00c170 con 0x7f6a8c0039c0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c00c370 con 0x7f6a8c004810 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 0x7f6a8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa37fe640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c0039c0 0x7f6a8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa2ffd640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c004810 0x7f6a8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 -- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c004810 msgr2=0x7f6a8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c004810 0x7f6a8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 -- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c0039c0 msgr2=0x7f6a8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c0039c0 0x7f6a8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa3fff640 1 -- 172.21.15.22:0/45063858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c003530 con 0x7f6a8c0051d0 2024-04-27T07:51:29.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.667+0000 7f6aa2ffd640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c004810 0x7f6a8c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:29.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa3fff640 1 -- 172.21.15.22:0/45063858 reap_dead start 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa3fff640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 0x7f6a8c00b9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40b7d60 tx=0x7f6a9c01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b033b0dd638b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c007d40 con 0x7f6a8c0051d0 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c005b00 con 0x7f6a8c0051d0 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa27fc640 1 -- 172.21.15.22:0/45063858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c007070 con 0x7f6a8c0051d0 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 msgr2=0x7f6a8c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 0x7f6a8c00b9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40b7d60 tx=0x7f6a9c01f3d0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- 172.21.15.22:0/45063858 shutdown_connections 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/45063858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a8c0051d0 0x7f6a8c00b9d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- 172.21.15.22:0/45063858 >> 172.21.15.22:0/45063858 conn(0x7f6aa4067d10 msgr2=0x7f6aa4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- 172.21.15.22:0/45063858 shutdown_connections 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- 172.21.15.22:0/45063858 wait complete. 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 Processor -- start 2024-04-27T07:51:29.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- start start 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c0051d0 0x7f6aa41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6aa4161bf0 0x7f6aa41f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40ba120 con 0x7f6aa4161770 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40d4260 con 0x7f6aa4161bf0 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa9e49640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa40ba2a0 con 0x7f6a8c0051d0 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58698/0 (socket says 172.21.15.22:58698) 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 -- 172.21.15.22:0/880058761 learned_addr learned my addr 172.21.15.22:0/880058761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa3fff640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6aa4161bf0 0x7f6aa41f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa37fe640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c0051d0 0x7f6aa41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 -- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c0051d0 msgr2=0x7f6aa41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c0051d0 0x7f6aa41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 -- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6aa4161bf0 msgr2=0x7f6aa41f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6aa4161bf0 0x7f6aa41f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 -- 172.21.15.22:0/880058761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa40b6bb0 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6aa2ffd640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a98001ec0 tx=0x7f6a9800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.671+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005480 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a98017070 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa40b5df0 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003860 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa40b63b0 con 0x7f6aa4161770 2024-04-27T07:51:29.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa4068c30 con 0x7f6aa4161770 2024-04-27T07:51:29.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a98003a00 con 0x7f6aa4161770 2024-04-27T07:51:29.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6a8bfff640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 0x7f6a9006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:29.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6a980866c0 con 0x7f6aa4161770 2024-04-27T07:51:29.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6aa37fe640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 0x7f6a9006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:29.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.675+0000 7f6aa37fe640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 0x7f6a9006a870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009fd0 tx=0x7f6a9400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:29.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.679+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6aa40b3d30 con 0x7f6aa4161770 2024-04-27T07:51:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.843+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aa415bf10 con 0x7f6aa4161770 2024-04-27T07:51:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.847+0000 7f6a8bfff640 1 -- 172.21.15.22:0/880058761 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f6a9800f2f0 con 0x7f6aa4161770 2024-04-27T07:51:29.853 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:29.853 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 msgr2=0x7f6a9006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 0x7f6a9006a870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009fd0 tx=0x7f6a9400b040 comp rx=0 tx=0).stop 2024-04-27T07:51:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 msgr2=0x7f6aa41f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a98001ec0 tx=0x7f6a9800d9d0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 shutdown_connections 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6a900683b0 0x7f6a9006a870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6aa4161bf0 0x7f6aa41f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa4161770 0x7f6aa41f7a90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 --2- 172.21.15.22:0/880058761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8c0051d0 0x7f6aa41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 >> 172.21.15.22:0/880058761 conn(0x7f6aa4067d10 msgr2=0x7f6aa40b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 shutdown_connections 2024-04-27T07:51:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.851+0000 7f6aa9e49640 1 -- 172.21.15.22:0/880058761 wait complete. 2024-04-27T07:51:29.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:29.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.991+0000 7f5716174640 1 Processor -- start 2024-04-27T07:51:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.991+0000 7f5716174640 1 -- start start 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f5716174640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5710071f70 0x7f5710072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f5716174640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f571006ee50 0x7f571006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f5716174640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5710072890 con 0x7f57100cffe0 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f5716174640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f57100ce0d0 con 0x7f5710071f70 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f5716174640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f57100cd8c0 con 0x7f571006ee50 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5710071f70 0x7f5710072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5710071f70 0x7f5710072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56912/0 (socket says 172.21.15.22:56912) 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 -- 172.21.15.22:0/435940226 learned_addr learned my addr 172.21.15.22:0/435940226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570ffff640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f571006ee50 0x7f571006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 -- 172.21.15.22:0/435940226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f571006ee50 msgr2=0x7f571006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f571006ee50 0x7f571006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 -- 172.21.15.22:0/435940226 >> v1:172.21.15.183:6789/0 conn(0x7f57100cffe0 legacy=0x7f571006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 -- 172.21.15.22:0/435940226 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100cdb10 con 0x7f5710071f70 2024-04-27T07:51:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570f7fe640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5710071f70 0x7f5710072350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfd5d933221a2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570effd640 1 -- 172.21.15.22:0/435940226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000b0070 con 0x7f5710071f70 2024-04-27T07:51:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570effd640 1 -- 172.21.15.22:0/435940226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57000ab600 con 0x7f5710071f70 2024-04-27T07:51:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.995+0000 7f570effd640 1 -- 172.21.15.22:0/435940226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000ab9d0 con 0x7f5710071f70 2024-04-27T07:51:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/435940226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5710071f70 msgr2=0x7f5710072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5710071f70 0x7f5710072350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570001c4d0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/435940226 shutdown_connections 2024-04-27T07:51:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f571006ee50 0x7f571006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- 172.21.15.22:0/435940226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5710071f70 0x7f5710072350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/435940226 >> 172.21.15.22:0/435940226 conn(0x7f57100b21d0 msgr2=0x7f57100b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/435940226 shutdown_connections 2024-04-27T07:51:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/435940226 wait complete. 2024-04-27T07:51:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 Processor -- start 2024-04-27T07:51:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- start start 2024-04-27T07:51:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f571006ee50 0x7f57101430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5710071f70 0x7f5710144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5710073130 con 0x7f57100cffe0 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5710072fb0 con 0x7f571006ee50 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f57100cceb0 con 0x7f5710071f70 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56920/0 (socket says 172.21.15.22:56920) 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 -- 172.21.15.22:0/791245469 learned_addr learned my addr 172.21.15.22:0/791245469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5706ffd640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5710071f70 0x7f5710144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570f7fe640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f571006ee50 0x7f57101430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 -- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5710071f70 msgr2=0x7f5710144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5710071f70 0x7f5710144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 -- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f571006ee50 msgr2=0x7f57101430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f571006ee50 0x7f57101430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 -- 172.21.15.22:0/791245469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5710142030 con 0x7f57100cffe0 2024-04-27T07:51:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570ffff640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e60 tx=0x7f570800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708014420 con 0x7f57100cffe0 2024-04-27T07:51:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5708019070 con 0x7f57100cffe0 2024-04-27T07:51:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5710142310 con 0x7f57100cffe0 2024-04-27T07:51:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708014820 con 0x7f57100cffe0 2024-04-27T07:51:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:29.999+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5710142bb0 con 0x7f57100cffe0 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57100ccce0 con 0x7f57100cffe0 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f57080149c0 con 0x7f57100cffe0 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f570cff9640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 0x7f56f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f57080871a0 con 0x7f57100cffe0 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f570f7fe640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 0x7f56f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.003+0000 7f570f7fe640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 0x7f56f006a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57000ad000 tx=0x7f57000a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.007+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57080577f0 con 0x7f57100cffe0 2024-04-27T07:51:30.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.171+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57101424e0 con 0x7f57100cffe0 2024-04-27T07:51:30.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.171+0000 7f570cff9640 1 -- 172.21.15.22:0/791245469 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f57080570f0 con 0x7f57100cffe0 2024-04-27T07:51:30.178 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 msgr2=0x7f56f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 0x7f56f006a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57000ad000 tx=0x7f57000a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 msgr2=0x7f571015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e60 tx=0x7f570800e4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 shutdown_connections 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f56f0068380 0x7f56f006a840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57100cffe0 0x7f571015c0f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5710071f70 0x7f5710144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 --2- 172.21.15.22:0/791245469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f571006ee50 0x7f57101430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 >> 172.21.15.22:0/791245469 conn(0x7f57100b21d0 msgr2=0x7f571015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 shutdown_connections 2024-04-27T07:51:30.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.175+0000 7f5716174640 1 -- 172.21.15.22:0/791245469 wait complete. 2024-04-27T07:51:30.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:30.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 Processor -- start 2024-04-27T07:51:30.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 -- start start 2024-04-27T07:51:30.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8ca4167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40cdb10 con 0x7f8ca4071f70 2024-04-27T07:51:30.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40cddf0 con 0x7f8ca40cffe0 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8caa7d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca40ce0d0 con 0x7f8ca406f750 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8ca4071f70 0x7f8ca4072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57502/0 (socket says 172.21.15.22:57502) 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 -- 172.21.15.22:0/2841021122 learned_addr learned my addr 172.21.15.22:0/2841021122 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca8d6f640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8ca4167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583376821 0 0) 0x7f8ca40cdb10 con 0x7f8ca4071f70 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c8c003530 con 0x7f8ca4071f70 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3168091907 0 0) 0x7f8c8c003530 con 0x7f8ca4071f70 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 msgr2=0x7f8ca4167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8ca4167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 >> v1:172.21.15.183:6789/0 conn(0x7f8ca40cffe0 legacy=0x7f8ca4164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca406e220 con 0x7f8ca4071f70 2024-04-27T07:51:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f8c94001680 con 0x7f8ca4071f70 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 >> v1:172.21.15.22:6789/0 conn(0x7f8ca4071f70 legacy=0x7f8ca4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8c0039f0 0x7f8c8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8c8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40ce0d0 con 0x7f8c8c0039f0 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c00bb00 con 0x7f8c8c004d50 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c00bd00 con 0x7f8ca406f750 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8c0039f0 0x7f8c8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca37fe640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8c8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca8d6f640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 -- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 msgr2=0x7f8c8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ca406f750 0x7f8c8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 -- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c8c004d50 msgr2=0x7f8c8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c8c004d50 0x7f8c8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.315+0000 7f8ca3fff640 1 -- 172.21.15.22:0/2841021122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c003530 con 0x7f8c8c0039f0 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8c0039f0 0x7f8c8c003e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c210 tx=0x7f8c9401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5645251e51fcb568 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94007d40 con 0x7f8c8c0039f0 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c940a9070 con 0x7f8c8c0039f0 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca2ffd640 1 -- 172.21.15.22:0/2841021122 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94005d40 con 0x7f8c8c0039f0 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- 172.21.15.22:0/2841021122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8c0039f0 msgr2=0x7f8c8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/2841021122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8c0039f0 0x7f8c8c003e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c210 tx=0x7f8c9401cbe0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca8d6f640 1 -- 172.21.15.22:0/2841021122 reap_dead start 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- 172.21.15.22:0/2841021122 shutdown_connections 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- 172.21.15.22:0/2841021122 >> 172.21.15.22:0/2841021122 conn(0x7f8ca40b21d0 msgr2=0x7f8ca40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- 172.21.15.22:0/2841021122 shutdown_connections 2024-04-27T07:51:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- 172.21.15.22:0/2841021122 wait complete. 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 Processor -- start 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- start start 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c8c0039f0 0x7f8ca415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4145060 0x7f8ca41454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca41682d0 con 0x7f8ca4141d00 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca406dfd0 con 0x7f8ca4145060 2024-04-27T07:51:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8caa7d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca4168450 con 0x7f8c8c0039f0 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56940/0 (socket says 172.21.15.22:56940) 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca37fe640 1 -- 172.21.15.22:0/4293377436 learned_addr learned my addr 172.21.15.22:0/4293377436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c8c0039f0 0x7f8ca415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.319+0000 7f8ca8d6f640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4145060 0x7f8ca41454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 -- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c8c0039f0 msgr2=0x7f8ca415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c8c0039f0 0x7f8ca415ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 -- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4145060 msgr2=0x7f8ca41454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4145060 0x7f8ca41454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 -- 172.21.15.22:0/4293377436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca415c230 con 0x7f8ca4141d00 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca37fe640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e4d0 tx=0x7f8c9800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c980053a0 con 0x7f8ca4141d00 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c98018070 con 0x7f8ca4141d00 2024-04-27T07:51:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98003820 con 0x7f8ca4141d00 2024-04-27T07:51:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca40b55f0 con 0x7f8ca4141d00 2024-04-27T07:51:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca415f6c0 con 0x7f8ca4141d00 2024-04-27T07:51:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca404ff10 con 0x7f8ca4141d00 2024-04-27T07:51:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8c980039c0 con 0x7f8ca4141d00 2024-04-27T07:51:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 0x7f8c9006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8c98086710 con 0x7f8ca4141d00 2024-04-27T07:51:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.323+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 0x7f8c9006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.327+0000 7f8ca3fff640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 0x7f8c9006a870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c940ab000 tx=0x7f8c940b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.327+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ca415c230 con 0x7f8ca4141d00 2024-04-27T07:51:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.491+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ca4072cb0 con 0x7f8ca4141d00 2024-04-27T07:51:30.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8ca0ff9640 1 -- 172.21.15.22:0/4293377436 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f8c98056cc0 con 0x7f8ca4141d00 2024-04-27T07:51:30.499 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 msgr2=0x7f8c9006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 0x7f8c9006a870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c940ab000 tx=0x7f8c940b1040 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 msgr2=0x7f8ca413b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e4d0 tx=0x7f8c9800e990 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 shutdown_connections 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8c900683b0 0x7f8c9006a870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4145060 0x7f8ca41454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ca4141d00 0x7f8ca413b770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 --2- 172.21.15.22:0/4293377436 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c8c0039f0 0x7f8ca415ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 >> 172.21.15.22:0/4293377436 conn(0x7f8ca40b21d0 msgr2=0x7f8ca40b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 shutdown_connections 2024-04-27T07:51:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.495+0000 7f8caa7d2640 1 -- 172.21.15.22:0/4293377436 wait complete. 2024-04-27T07:51:30.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:30.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.643+0000 7ff418479640 1 Processor -- start 2024-04-27T07:51:30.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.643+0000 7ff418479640 1 -- start start 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff418479640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff410072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff418479640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4100b5390 con 0x7ff4100b4ee0 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff418479640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4100b5670 con 0x7ff4100cee20 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff418479640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100ce340 con 0x7ff41015ecd0 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415a14640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff4100b4ee0 0x7ff4100b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57504/0 (socket says 172.21.15.22:57504) 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415a14640 1 -- 172.21.15.22:0/1221534913 learned_addr learned my addr 172.21.15.22:0/1221534913 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff410072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088629077 0 0) 0x7ff4100b5390 con 0x7ff4100b4ee0 2024-04-27T07:51:30.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3fc003530 con 0x7ff4100b4ee0 2024-04-27T07:51:30.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2354495747 0 0) 0x7ff3fc003530 con 0x7ff4100b4ee0 2024-04-27T07:51:30.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 msgr2=0x7ff410072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff410072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 >> v1:172.21.15.183:6789/0 conn(0x7ff4100cee20 legacy=0x7ff41006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4100ce620 con 0x7ff4100b4ee0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7ff400001680 con 0x7ff4100b4ee0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 >> v1:172.21.15.22:6789/0 conn(0x7ff4100b4ee0 legacy=0x7ff4100b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3fc0039f0 0x7ff3fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc004810 0x7ff3fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff3fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100ce340 con 0x7ff3fc0039f0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc00ba80 con 0x7ff3fc004810 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff3fc00de90 con 0x7ff41015ecd0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3fc0039f0 0x7ff3fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415a14640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc004810 0x7ff3fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416a16640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff3fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 -- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 msgr2=0x7ff3fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff3fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 -- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc004810 msgr2=0x7ff3fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc004810 0x7ff3fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416215640 1 -- 172.21.15.22:0/1221534913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc003530 con 0x7ff3fc0039f0 2024-04-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff416a16640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41015ecd0 0x7ff3fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.647+0000 7ff415a14640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc004810 0x7ff3fc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416215640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3fc0039f0 0x7ff3fc003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009f90 tx=0x7ff40c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c537e5f6bf2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0b2070 con 0x7ff3fc0039f0 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c0a5b00 con 0x7ff3fc0039f0 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff415213640 1 -- 172.21.15.22:0/1221534913 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c009c10 con 0x7ff3fc0039f0 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- 172.21.15.22:0/1221534913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3fc0039f0 msgr2=0x7ff3fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 --2- 172.21.15.22:0/1221534913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3fc0039f0 0x7ff3fc003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009f90 tx=0x7ff40c01cbf0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416a16640 1 -- 172.21.15.22:0/1221534913 reap_dead start 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- 172.21.15.22:0/1221534913 shutdown_connections 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- 172.21.15.22:0/1221534913 >> 172.21.15.22:0/1221534913 conn(0x7ff4100b21d0 msgr2=0x7ff4100b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- 172.21.15.22:0/1221534913 shutdown_connections 2024-04-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- 172.21.15.22:0/1221534913 wait complete. 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 Processor -- start 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- start start 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 0x7ff41014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41014dc10 0x7ff41014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100cf230 con 0x7ff4101f7c80 2024-04-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff41006fa00 con 0x7ff3fc0039f0 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff418479640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100cf3b0 con 0x7ff41014dc10 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416a16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416a16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56954/0 (socket says 172.21.15.22:56954) 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416a16640 1 -- 172.21.15.22:0/4292038653 learned_addr learned my addr 172.21.15.22:0/4292038653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.651+0000 7ff416215640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 0x7ff41014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff415a14640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41014dc10 0x7ff41014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 -- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41014dc10 msgr2=0x7ff41014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41014dc10 0x7ff41014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 -- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 msgr2=0x7ff41014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 0x7ff41014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 -- 172.21.15.22:0/4292038653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41014e0f0 con 0x7ff4101f7c80 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416215640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 0x7ff41014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416a16640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff41006ddc0 tx=0x7ff40800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408007030 con 0x7ff4101f7c80 2024-04-27T07:51:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4080071f0 con 0x7ff4101f7c80 2024-04-27T07:51:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41014e3d0 con 0x7ff4101f7c80 2024-04-27T07:51:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4080075e0 con 0x7ff4101f7c80 2024-04-27T07:51:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41014e990 con 0x7ff4101f7c80 2024-04-27T07:51:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41004ff10 con 0x7ff4101f7c80 2024-04-27T07:51:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff40801b020 con 0x7ff4101f7c80 2024-04-27T07:51:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 0x7ff3f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff4080862e0 con 0x7ff4101f7c80 2024-04-27T07:51:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.655+0000 7ff416215640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 0x7ff3f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.659+0000 7ff416215640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 0x7ff3f406a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0af000 tx=0x7ff40c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.659+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff408056930 con 0x7ff4101f7c80 2024-04-27T07:51:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4100700f0 con 0x7ff4101f7c80 2024-04-27T07:51:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff406ffd640 1 -- 172.21.15.22:0/4292038653 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff408056230 con 0x7ff4101f7c80 2024-04-27T07:51:30.828 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 msgr2=0x7ff3f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 0x7ff3f406a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0af000 tx=0x7ff40c0a7040 comp rx=0 tx=0).stop 2024-04-27T07:51:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 msgr2=0x7ff4101f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff41006ddc0 tx=0x7ff40800d890 comp rx=0 tx=0).stop 2024-04-27T07:51:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 shutdown_connections 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff3f4068380 0x7ff3f406a840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4101f7c80 0x7ff4101f8100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff41014dc10 0x7ff41014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 --2- 172.21.15.22:0/4292038653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc0039f0 0x7ff41014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 >> 172.21.15.22:0/4292038653 conn(0x7ff4100b21d0 msgr2=0x7ff4100b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 shutdown_connections 2024-04-27T07:51:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.823+0000 7ff418479640 1 -- 172.21.15.22:0/4292038653 wait complete. 2024-04-27T07:51:30.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:30.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 Processor -- start 2024-04-27T07:51:30.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 -- start start 2024-04-27T07:51:30.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa3641454b0 0x7fa364072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 0x7fa36406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa36406e050 con 0x7fa364072030 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa36406f330 con 0x7fa3641454b0 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36c202640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa3640ceae0 con 0x7fa364145a00 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36a79f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa364072030 0x7fa36415cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57508/0 (socket says 172.21.15.22:57508) 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa369f9e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa3641454b0 0x7fa364072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36a79f640 1 -- 172.21.15.22:0/3174794617 learned_addr learned my addr 172.21.15.22:0/3174794617 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36979d640 1 --2- 172.21.15.22:0/3174794617 >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 0x7fa36406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa368f9c640 1 -- 172.21.15.22:0/3174794617 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198096952 0 0) 0x7fa36406e050 con 0x7fa364072030 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa368f9c640 1 -- 172.21.15.22:0/3174794617 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa350003530 con 0x7fa364072030 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa369f9e640 1 -- 172.21.15.22:0/3174794617 >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 msgr2=0x7fa36406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa369f9e640 1 --2- 172.21.15.22:0/3174794617 >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 0x7fa36406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa369f9e640 1 -- 172.21.15.22:0/3174794617 >> v1:172.21.15.22:6789/0 conn(0x7fa364072030 legacy=0x7fa36415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa369f9e640 1 -- 172.21.15.22:0/3174794617 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3640cedc0 con 0x7fa3641454b0 2024-04-27T07:51:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.971+0000 7fa36979d640 1 --2- 172.21.15.22:0/3174794617 >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 0x7fa36406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:30.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.975+0000 7fa369f9e640 1 --2- 172.21.15.22:0/3174794617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa364072fa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa3640d0310 tx=0x7fa36001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97209507dbb092e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.975+0000 7fa368f9c640 1 -- 172.21.15.22:0/3174794617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3600b0070 con 0x7fa3641454b0 2024-04-27T07:51:30.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.975+0000 7fa368f9c640 1 -- 172.21.15.22:0/3174794617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3600ab600 con 0x7fa3641454b0 2024-04-27T07:51:30.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.975+0000 7fa368f9c640 1 -- 172.21.15.22:0/3174794617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3600ab9d0 con 0x7fa3641454b0 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/3174794617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 msgr2=0x7fa364072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- 172.21.15.22:0/3174794617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa364072fa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa3640d0310 tx=0x7fa36001c480 comp rx=0 tx=0).stop 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/3174794617 shutdown_connections 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- 172.21.15.22:0/3174794617 >> v2:172.21.15.183:3300/0 conn(0x7fa364145a00 0x7fa36406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- 172.21.15.22:0/3174794617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa364072fa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/3174794617 >> 172.21.15.22:0/3174794617 conn(0x7fa364067f10 msgr2=0x7fa36406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/3174794617 shutdown_connections 2024-04-27T07:51:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/3174794617 wait complete. 2024-04-27T07:51:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 Processor -- start 2024-04-27T07:51:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- start start 2024-04-27T07:51:30.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 0x7fa36415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa364145a00 0x7fa364141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa364072850 con 0x7fa3641454b0 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa36415d820 con 0x7fa364072030 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa36413d640 con 0x7fa364145a00 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56972/0 (socket says 172.21.15.22:56972) 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 -- 172.21.15.22:0/316243350 learned_addr learned my addr 172.21.15.22:0/316243350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa369f9e640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 0x7fa36415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 -- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa364145a00 msgr2=0x7fa364141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36a79f640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa364145a00 0x7fa364141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa364145a00 0x7fa364141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 -- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 msgr2=0x7fa36415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 0x7fa36415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 -- 172.21.15.22:0/316243350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36413d920 con 0x7fa3641454b0 2024-04-27T07:51:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa369f9e640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 0x7fa36415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36979d640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa35800d830 tx=0x7fa35800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358018020 con 0x7fa3641454b0 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3580096c0 con 0x7fa3641454b0 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36413dc00 con 0x7fa3641454b0 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358009ab0 con 0x7fa3641454b0 2024-04-27T07:51:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.979+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36413f500 con 0x7fa3641454b0 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa364068c30 con 0x7fa3641454b0 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa35801b020 con 0x7fa3641454b0 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa3567fc640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 0x7fa34806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3580871d0 con 0x7fa3641454b0 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa369f9e640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 0x7fa34806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.983+0000 7fa369f9e640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 0x7fa34806a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3600ad000 tx=0x7fa360008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:30.987+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa358022020 con 0x7fa3641454b0 2024-04-27T07:51:31.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.159+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa36406e050 con 0x7fa3641454b0 2024-04-27T07:51:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa3567fc640 1 -- 172.21.15.22:0/316243350 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa358057790 con 0x7fa3641454b0 2024-04-27T07:51:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 msgr2=0x7fa34806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 0x7fa34806a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3600ad000 tx=0x7fa360008040 comp rx=0 tx=0).stop 2024-04-27T07:51:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 msgr2=0x7fa36413ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa35800d830 tx=0x7fa35800dcf0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 shutdown_connections 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa348068380 0x7fa34806a840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa364145a00 0x7fa364141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3641454b0 0x7fa36413ccd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 --2- 172.21.15.22:0/316243350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa364072030 0x7fa36415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 >> 172.21.15.22:0/316243350 conn(0x7fa364067f10 msgr2=0x7fa364144710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 shutdown_connections 2024-04-27T07:51:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.163+0000 7fa36c202640 1 -- 172.21.15.22:0/316243350 wait complete. 2024-04-27T07:51:31.185 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-04-27T07:51:31.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:31.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-27T07:51:31.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:31.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:51:31.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:31.211 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-27T07:51:31.265 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:31.265 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:31.272 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:31.272 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:31.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 Processor -- start 2024-04-27T07:51:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 -- start start 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f016c144e40 0x7f016c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c145220 con 0x7f016c06ef10 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c162df0 con 0x7f016c06f560 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f0172505640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f016c1630d0 con 0x7f016c144e40 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f016c144e40 0x7f016c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f016c06ef10 0x7f016c1623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57516/0 (socket says 172.21.15.22:57516) 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016b7fe640 1 -- 172.21.15.22:0/3635717229 learned_addr learned my addr 172.21.15.22:0/3635717229 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016bfff640 1 -- 172.21.15.22:0/3635717229 >> v1:172.21.15.183:6789/0 conn(0x7f016c06f560 legacy=0x7f016c0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016bfff640 1 -- 172.21.15.22:0/3635717229 >> v1:172.21.15.22:6789/0 conn(0x7f016c06ef10 legacy=0x7f016c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016bfff640 1 -- 172.21.15.22:0/3635717229 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c06e1d0 con 0x7f016c144e40 2024-04-27T07:51:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.439+0000 7f016affd640 1 -- 172.21.15.22:0/3635717229 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523035011 0 0) 0x7f015c001120 con 0x7f016c06ef10 2024-04-27T07:51:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f016bfff640 1 --2- 172.21.15.22:0/3635717229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c06e9d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01540097c0 tx=0x7f015401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cc8f8d78989b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f016affd640 1 -- 172.21.15.22:0/3635717229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01540b1070 con 0x7f016c144e40 2024-04-27T07:51:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f016affd640 1 -- 172.21.15.22:0/3635717229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01540ac410 con 0x7f016c144e40 2024-04-27T07:51:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f016affd640 1 -- 172.21.15.22:0/3635717229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01540ac7e0 con 0x7f016c144e40 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 -- 172.21.15.22:0/3635717229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 msgr2=0x7f016c06e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 --2- 172.21.15.22:0/3635717229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c06e9d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01540097c0 tx=0x7f015401c4a0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 -- 172.21.15.22:0/3635717229 shutdown_connections 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 --2- 172.21.15.22:0/3635717229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c06e9d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 -- 172.21.15.22:0/3635717229 >> 172.21.15.22:0/3635717229 conn(0x7f016c067f10 msgr2=0x7f016c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 -- 172.21.15.22:0/3635717229 shutdown_connections 2024-04-27T07:51:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.443+0000 7f0172505640 1 -- 172.21.15.22:0/3635717229 wait complete. 2024-04-27T07:51:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 Processor -- start 2024-04-27T07:51:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- start start 2024-04-27T07:51:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 0x7f016c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f016c13b940 0x7f016c13bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f016c072370 con 0x7f016c144e40 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0721f0 con 0x7f016c06f560 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f016c16cd50 con 0x7f016c13b940 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56990/0 (socket says 172.21.15.22:56990) 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 -- 172.21.15.22:0/4212852761 learned_addr learned my addr 172.21.15.22:0/4212852761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016bfff640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 0x7f016c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0170aa2640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f016c13b940 0x7f016c13bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 -- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f016c13b940 msgr2=0x7f016c13bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f016c13b940 0x7f016c13bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 -- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 msgr2=0x7f016c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 0x7f016c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 -- 172.21.15.22:0/4212852761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1414f0 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016bfff640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 0x7f016c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f016b7fe640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c780 tx=0x7f015c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c018070 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c0072d0 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1417d0 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c0047f0 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c141d90 con 0x7f016c144e40 2024-04-27T07:51:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c068c30 con 0x7f016c144e40 2024-04-27T07:51:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.447+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f015c01b070 con 0x7f016c144e40 2024-04-27T07:51:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.451+0000 7f0168ff9640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 0x7f014c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.451+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f015c085e10 con 0x7f016c144e40 2024-04-27T07:51:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.451+0000 7f016bfff640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 0x7f014c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.451+0000 7f016bfff640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 0x7f014c06a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01540ae000 tx=0x7f0154008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.455+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f015c089030 con 0x7f016c144e40 2024-04-27T07:51:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.615+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f016c06d010 con 0x7f016c144e40 2024-04-27T07:51:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0168ff9640 1 -- 172.21.15.22:0/4212852761 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f016c145220 con 0x7f016c144e40 2024-04-27T07:51:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:31.624 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 msgr2=0x7f014c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 0x7f014c06a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01540ae000 tx=0x7f0154008040 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 msgr2=0x7f016c143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c780 tx=0x7f015c00cc40 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 shutdown_connections 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f014c068380 0x7f014c06a840 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f016c13b940 0x7f016c13bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f016c144e40 0x7f016c143bf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 --2- 172.21.15.22:0/4212852761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f016c06f560 0x7f016c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 >> 172.21.15.22:0/4212852761 conn(0x7f016c067f10 msgr2=0x7f016c15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 shutdown_connections 2024-04-27T07:51:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.619+0000 7f0172505640 1 -- 172.21.15.22:0/4212852761 wait complete. 2024-04-27T07:51:31.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:31.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 Processor -- start 2024-04-27T07:51:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 -- start start 2024-04-27T07:51:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efc600b51b0 0x7efc600b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc600b3e00 0x7efc600aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc600b5ad0 con 0x7efc60067910 2024-04-27T07:51:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efc6006ec60 con 0x7efc600b3e00 2024-04-27T07:51:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc65db4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7efc6006ef40 con 0x7efc600b51b0 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc600b3e00 0x7efc600aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc600b3e00 0x7efc600aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56998/0 (socket says 172.21.15.22:56998) 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 -- 172.21.15.22:0/3314059918 learned_addr learned my addr 172.21.15.22:0/3314059918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 -- 172.21.15.22:0/3314059918 >> v2:172.21.15.183:3300/0 conn(0x7efc600b51b0 msgr2=0x7efc600b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5f7fe640 1 --2- 172.21.15.22:0/3314059918 >> v2:172.21.15.183:3300/0 conn(0x7efc600b51b0 0x7efc600b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 --2- 172.21.15.22:0/3314059918 >> v2:172.21.15.183:3300/0 conn(0x7efc600b51b0 0x7efc600b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 -- 172.21.15.22:0/3314059918 >> v1:172.21.15.183:6789/0 conn(0x7efc60067910 legacy=0x7efc6006fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.759+0000 7efc5ffff640 1 -- 172.21.15.22:0/3314059918 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6006f220 con 0x7efc600b3e00 2024-04-27T07:51:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc5ffff640 1 --2- 172.21.15.22:0/3314059918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b3e00 0x7efc600aec20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc600b3bc0 tx=0x7efc5801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9a95d05f5a655b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc5e7fc640 1 -- 172.21.15.22:0/3314059918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580a5790 con 0x7efc600b3e00 2024-04-27T07:51:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc5e7fc640 1 -- 172.21.15.22:0/3314059918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580a5930 con 0x7efc600b3e00 2024-04-27T07:51:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc5e7fc640 1 -- 172.21.15.22:0/3314059918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580a5d00 con 0x7efc600b3e00 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- 172.21.15.22:0/3314059918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b3e00 msgr2=0x7efc600aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- 172.21.15.22:0/3314059918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b3e00 0x7efc600aec20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc600b3bc0 tx=0x7efc5801c8d0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- 172.21.15.22:0/3314059918 shutdown_connections 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- 172.21.15.22:0/3314059918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b3e00 0x7efc600aec20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- 172.21.15.22:0/3314059918 >> v2:172.21.15.183:3300/0 conn(0x7efc600b51b0 0x7efc600b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- 172.21.15.22:0/3314059918 >> 172.21.15.22:0/3314059918 conn(0x7efc60073790 msgr2=0x7efc6006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- 172.21.15.22:0/3314059918 shutdown_connections 2024-04-27T07:51:31.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- 172.21.15.22:0/3314059918 wait complete. 2024-04-27T07:51:31.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 Processor -- start 2024-04-27T07:51:31.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- start start 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 0x7efc601faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc600b3e00 0x7efc60160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc6015ed00 con 0x7efc600b51b0 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc6015eb80 con 0x7efc600b3e00 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.763+0000 7efc65db4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efc6016b810 con 0x7efc60067910 2024-04-27T07:51:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57012/0 (socket says 172.21.15.22:57012) 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 -- 172.21.15.22:0/432519329 learned_addr learned my addr 172.21.15.22:0/432519329 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5effd640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc600b3e00 0x7efc60160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5f7fe640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 0x7efc601faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 -- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 msgr2=0x7efc601faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 0x7efc601faa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 -- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc600b3e00 msgr2=0x7efc60160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc600b3e00 0x7efc60160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 -- 172.21.15.22:0/432519329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc60067d40 con 0x7efc600b51b0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5f7fe640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 0x7efc601faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5ffff640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc580af040 tx=0x7efc58019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580bc020 con 0x7efc600b51b0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580b8670 con 0x7efc600b51b0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6015f340 con 0x7efc600b51b0 2024-04-27T07:51:31.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580b8a60 con 0x7efc600b51b0 2024-04-27T07:51:31.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6015f900 con 0x7efc600b51b0 2024-04-27T07:51:31.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6004ff10 con 0x7efc600b51b0 2024-04-27T07:51:31.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efc580bf020 con 0x7efc600b51b0 2024-04-27T07:51:31.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 0x7efc4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:31.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efc58121a90 con 0x7efc600b51b0 2024-04-27T07:51:31.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.767+0000 7efc5f7fe640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 0x7efc4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:31.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.771+0000 7efc5f7fe640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 0x7efc4c06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc50006fd0 tx=0x7efc50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:31.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.771+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efc580c6030 con 0x7efc600b51b0 2024-04-27T07:51:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.939+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc600d0ac0 con 0x7efc600b51b0 2024-04-27T07:51:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.939+0000 7efc47fff640 1 -- 172.21.15.22:0/432519329 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7efc580165e0 con 0x7efc600b51b0 2024-04-27T07:51:31.945 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 msgr2=0x7efc4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 0x7efc4c06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc50006fd0 tx=0x7efc50008040 comp rx=0 tx=0).stop 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 msgr2=0x7efc6014a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc580af040 tx=0x7efc58019670 comp rx=0 tx=0).stop 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 shutdown_connections 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7efc4c068380 0x7efc4c06a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc600b51b0 0x7efc6014a250 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc600b3e00 0x7efc60160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 --2- 172.21.15.22:0/432519329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc60067910 0x7efc601faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 >> 172.21.15.22:0/432519329 conn(0x7efc60073790 msgr2=0x7efc601518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 shutdown_connections 2024-04-27T07:51:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:31.943+0000 7efc65db4640 1 -- 172.21.15.22:0/432519329 wait complete. 2024-04-27T07:51:31.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:32.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 Processor -- start 2024-04-27T07:51:32.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 -- start start 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1b80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b80ced30 con 0x7fb1b80b05b0 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b8070130 con 0x7fb1b80b0b00 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bf79d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80d1150 con 0x7fb1b80d0830 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb1b80b05b0 0x7fb1b80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57520/0 (socket says 172.21.15.22:57520) 2024-04-27T07:51:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/3908128622 learned_addr learned my addr 172.21.15.22:0/3908128622 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bd539640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1b80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734378794 0 0) 0x7fb1b80ced30 con 0x7fb1b80b05b0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1a4003530 con 0x7fb1b80b05b0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1255385133 0 0) 0x7fb1a4003530 con 0x7fb1b80b05b0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 msgr2=0x7fb1b80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1b80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 >> v1:172.21.15.183:6789/0 conn(0x7fb1b80b0b00 legacy=0x7fb1b80cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b806bbd0 con 0x7fb1b80b05b0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fb1b0001680 con 0x7fb1b80b05b0 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 >> v1:172.21.15.22:6789/0 conn(0x7fb1b80b05b0 legacy=0x7fb1b80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a40039f0 0x7fb1a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a4004d50 0x7fb1a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80d1150 con 0x7fb1a4004d50 2024-04-27T07:51:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400b9d0 con 0x7fb1a40039f0 2024-04-27T07:51:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a400bbd0 con 0x7fb1b80d0830 2024-04-27T07:51:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a4004d50 0x7fb1a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bd539640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a40039f0 0x7fb1a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bcd38640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 msgr2=0x7fb1a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b80d0830 0x7fb1a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a40039f0 msgr2=0x7fb1a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a40039f0 0x7fb1a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/3908128622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a4003530 con 0x7fb1a4004d50 2024-04-27T07:51:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.091+0000 7fb1bd539640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a40039f0 0x7fb1a4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:32.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a4004d50 0x7fb1a40051d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c290 tx=0x7fb1b001f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c32b9009b555bde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007d60 con 0x7fb1a4004d50 2024-04-27T07:51:32.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b0005b00 con 0x7fb1a4004d50 2024-04-27T07:51:32.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1affff640 1 -- 172.21.15.22:0/3908128622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007070 con 0x7fb1a4004d50 2024-04-27T07:51:32.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 -- 172.21.15.22:0/3908128622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a4004d50 msgr2=0x7fb1a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/3908128622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a4004d50 0x7fb1a40051d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c290 tx=0x7fb1b001f450 comp rx=0 tx=0).stop 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/3908128622 reap_dead start 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 -- 172.21.15.22:0/3908128622 shutdown_connections 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 -- 172.21.15.22:0/3908128622 >> 172.21.15.22:0/3908128622 conn(0x7fb1b809aa30 msgr2=0x7fb1b80d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 -- 172.21.15.22:0/3908128622 shutdown_connections 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.095+0000 7fb1bf79d640 1 -- 172.21.15.22:0/3908128622 wait complete. 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 Processor -- start 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- start start 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b81564e0 0x7fb1b809b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1b81568c0 0x7fb1b809b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b806c170 con 0x7fb1b809d210 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b806be90 con 0x7fb1b81568c0 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b806c450 con 0x7fb1b81564e0 2024-04-27T07:51:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57028/0 (socket says 172.21.15.22:57028) 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/583817889 learned_addr learned my addr 172.21.15.22:0/583817889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bd539640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b81564e0 0x7fb1b809b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bcd38640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1b81568c0 0x7fb1b809b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b81564e0 msgr2=0x7fb1b809b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b81564e0 0x7fb1b809b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1b81568c0 msgr2=0x7fb1b809b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1b81568c0 0x7fb1b809b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 -- 172.21.15.22:0/583817889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b809e000 con 0x7fb1b809d210 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bdd3a640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c290 tx=0x7fb1a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00b5070 con 0x7fb1b809d210 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b0004ab0 con 0x7fb1b809d210 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b809e2e0 con 0x7fb1b809d210 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007d60 con 0x7fb1b809d210 2024-04-27T07:51:32.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81580d0 con 0x7fb1b809d210 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b804ff10 con 0x7fb1b809d210 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1b0007070 con 0x7fb1b809d210 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 0x7fb19c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.099+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1b01237e0 con 0x7fb1b809d210 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.103+0000 7fb1bd539640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 0x7fb19c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.103+0000 7fb1bd539640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 0x7fb19c06a870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1b809ea70 tx=0x7fb1a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.107+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1b0019160 con 0x7fb1b809d210 2024-04-27T07:51:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.267+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1b80cdcf0 con 0x7fb1b809d210 2024-04-27T07:51:32.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.267+0000 7fb1adffb640 1 -- 172.21.15.22:0/583817889 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fb1b00f3e60 con 0x7fb1b809d210 2024-04-27T07:51:32.275 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 msgr2=0x7fb19c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 0x7fb19c06a870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1b809ea70 tx=0x7fb1a8008040 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 msgr2=0x7fb1b809d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c290 tx=0x7fb1a4005840 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 shutdown_connections 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb19c0683b0 0x7fb19c06a870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1b809d210 0x7fb1b809d6c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1b81568c0 0x7fb1b809b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 --2- 172.21.15.22:0/583817889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1b81564e0 0x7fb1b809b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 >> 172.21.15.22:0/583817889 conn(0x7fb1b809aa30 msgr2=0x7fb1b80d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 shutdown_connections 2024-04-27T07:51:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.271+0000 7fb1bf79d640 1 -- 172.21.15.22:0/583817889 wait complete. 2024-04-27T07:51:32.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:32.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 Processor -- start 2024-04-27T07:51:32.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 -- start start 2024-04-27T07:51:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 0x7fccc41668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccc40cdb10 con 0x7fccc4071f70 2024-04-27T07:51:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fccc40ce0d0 con 0x7fccc406f750 2024-04-27T07:51:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fcccad12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fccc40cd8c0 con 0x7fccc40cffe0 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:35496/0 (socket says 172.21.15.22:35496) 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc92af640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 0x7fccc41668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 -- 172.21.15.22:0/1776469346 learned_addr learned my addr 172.21.15.22:0/1776469346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 -- 172.21.15.22:0/1776469346 >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 msgr2=0x7fccc41668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 --2- 172.21.15.22:0/1776469346 >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 0x7fccc41668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 -- 172.21.15.22:0/1776469346 >> v1:172.21.15.183:6789/0 conn(0x7fccc4071f70 legacy=0x7fccc4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc3fff640 1 -- 172.21.15.22:0/1776469346 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc40cddf0 con 0x7fccc40cffe0 2024-04-27T07:51:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.419+0000 7fccc92af640 1 --2- 172.21.15.22:0/1776469346 >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 0x7fccc41668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fccc3fff640 1 --2- 172.21.15.22:0/1776469346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009870 tx=0x7fccb801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a713321f1ebf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fccc37fe640 1 -- 172.21.15.22:0/1776469346 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80b0070 con 0x7fccc40cffe0 2024-04-27T07:51:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fccc37fe640 1 -- 172.21.15.22:0/1776469346 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb80ab600 con 0x7fccc40cffe0 2024-04-27T07:51:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fccc37fe640 1 -- 172.21.15.22:0/1776469346 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80ab9d0 con 0x7fccc40cffe0 2024-04-27T07:51:32.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 -- 172.21.15.22:0/1776469346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 msgr2=0x7fccc4164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 --2- 172.21.15.22:0/1776469346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009870 tx=0x7fccb801c4d0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 -- 172.21.15.22:0/1776469346 shutdown_connections 2024-04-27T07:51:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 --2- 172.21.15.22:0/1776469346 >> v2:172.21.15.183:3300/0 conn(0x7fccc406f750 0x7fccc41668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 --2- 172.21.15.22:0/1776469346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc40cffe0 0x7fccc4164390 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 -- 172.21.15.22:0/1776469346 >> 172.21.15.22:0/1776469346 conn(0x7fccc40b21d0 msgr2=0x7fccc40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 -- 172.21.15.22:0/1776469346 shutdown_connections 2024-04-27T07:51:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.423+0000 7fcccad12640 1 -- 172.21.15.22:0/1776469346 wait complete. 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 Processor -- start 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- start start 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc406f750 0x7fccc4143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccc40cffe0 0x7fccc4160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fccc40cd030 con 0x7fccc4071f70 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccc40cceb0 con 0x7fccc40cffe0 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fccc4169640 con 0x7fccc406f750 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57034/0 (socket says 172.21.15.22:57034) 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 -- 172.21.15.22:0/1618090408 learned_addr learned my addr 172.21.15.22:0/1618090408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc92af640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccc40cffe0 0x7fccc4160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc8aae640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc406f750 0x7fccc4143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 -- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc406f750 msgr2=0x7fccc4143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc406f750 0x7fccc4143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 -- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccc40cffe0 msgr2=0x7fccc4160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccc40cffe0 0x7fccc4160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 -- 172.21.15.22:0/1618090408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc40ac500 con 0x7fccc4071f70 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc3fff640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccb80ae040 tx=0x7fccb8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80b9070 con 0x7fccc4071f70 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb80b4430 con 0x7fccc4071f70 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc40ac7e0 con 0x7fccc4071f70 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80b0040 con 0x7fccc4071f70 2024-04-27T07:51:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc40acda0 con 0x7fccc4071f70 2024-04-27T07:51:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc404ff10 con 0x7fccc4071f70 2024-04-27T07:51:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.427+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fccb80bc070 con 0x7fccc4071f70 2024-04-27T07:51:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.431+0000 7fccc17fa640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 0x7fccb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.431+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fccb811fb70 con 0x7fccc4071f70 2024-04-27T07:51:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.431+0000 7fccc8aae640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 0x7fccb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.431+0000 7fccc8aae640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 0x7fccb006a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccb4001650 tx=0x7fccb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.435+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fccb8016150 con 0x7fccc4071f70 2024-04-27T07:51:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccc41457d0 con 0x7fccc4071f70 2024-04-27T07:51:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fccc17fa640 1 -- 172.21.15.22:0/1618090408 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fccb80efae0 con 0x7fccc4071f70 2024-04-27T07:51:32.604 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 msgr2=0x7fccb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 0x7fccb006a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccb4001650 tx=0x7fccb4009040 comp rx=0 tx=0).stop 2024-04-27T07:51:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 msgr2=0x7fccc415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccb80ae040 tx=0x7fccb8019270 comp rx=0 tx=0).stop 2024-04-27T07:51:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 shutdown_connections 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fccb0068380 0x7fccb006a840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccc40cffe0 0x7fccc4160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc4071f70 0x7fccc415c110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 --2- 172.21.15.22:0/1618090408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccc406f750 0x7fccc4143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 >> 172.21.15.22:0/1618090408 conn(0x7fccc40b21d0 msgr2=0x7fccc4144480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 shutdown_connections 2024-04-27T07:51:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.599+0000 7fcccad12640 1 -- 172.21.15.22:0/1618090408 wait complete. 2024-04-27T07:51:32.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:32.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.739+0000 7f7d0d0c8640 1 Processor -- start 2024-04-27T07:51:32.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.739+0000 7f7d0d0c8640 1 -- start start 2024-04-27T07:51:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0d0c8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7d08072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0d0c8640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0806e050 con 0x7f7d081454b0 2024-04-27T07:51:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0d0c8640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d080ceae0 con 0x7f7d08072740 2024-04-27T07:51:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0d0c8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080cedc0 con 0x7f7d08145a00 2024-04-27T07:51:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d06d9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7d081454b0 0x7f7d08072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57532/0 (socket says 172.21.15.22:57532) 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d06d9d640 1 -- 172.21.15.22:0/2549280408 learned_addr learned my addr 172.21.15.22:0/2549280408 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0759e640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7d08072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 163010737 0 0) 0x7f7d0806e050 con 0x7f7d081454b0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf0003530 con 0x7f7d081454b0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2868261676 0 0) 0x7f7cf0003530 con 0x7f7d081454b0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 msgr2=0x7f7d08072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7d08072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 >> v1:172.21.15.183:6789/0 conn(0x7f7d08072740 legacy=0x7f7d0815cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0806e310 con 0x7f7d081454b0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7cf8001680 con 0x7f7d081454b0 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 >> v1:172.21.15.22:6789/0 conn(0x7f7d081454b0 legacy=0x7f7d08072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cf00039f0 0x7f7cf0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7cf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7cf00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080cedc0 con 0x7f7cf0004810 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf000ba80 con 0x7f7cf00039f0 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf000bc80 con 0x7f7d08145a00 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7cf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d06d9d640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cf00039f0 0x7f7cf0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0759e640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7cf00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 -- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 msgr2=0x7f7cf00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7cf00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 -- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cf00039f0 msgr2=0x7f7cf0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cf00039f0 0x7f7cf0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0659c640 1 -- 172.21.15.22:0/2549280408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf0003530 con 0x7f7cf0004810 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d0759e640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d08145a00 0x7f7cf00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.743+0000 7f7d06d9d640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cf00039f0 0x7f7cf0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0659c640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7cf0004c90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc001750 tx=0x7f7cfc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc1a3cbc6ccd73f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0013a0 con 0x7f7cf0004810 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0052f0 con 0x7f7cf0004810 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d05d9b640 1 -- 172.21.15.22:0/2549280408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc003840 con 0x7f7cf0004810 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/2549280408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 msgr2=0x7f7cf0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/2549280408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7cf0004c90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc001750 tx=0x7f7cfc01c3c0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0759e640 1 -- 172.21.15.22:0/2549280408 reap_dead start 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/2549280408 shutdown_connections 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/2549280408 >> 172.21.15.22:0/2549280408 conn(0x7f7d08067f10 msgr2=0x7f7d0806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/2549280408 shutdown_connections 2024-04-27T07:51:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/2549280408 wait complete. 2024-04-27T07:51:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 Processor -- start 2024-04-27T07:51:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- start start 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0813c470 0x7f7d0815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0813cc40 0x7f7d0815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08072fa0 con 0x7f7cf0004810 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0806e8d0 con 0x7f7d0813c470 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d0d0c8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7d08073120 con 0x7f7d0813cc40 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57056/0 (socket says 172.21.15.22:57056) 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.747+0000 7f7d06d9d640 1 -- 172.21.15.22:0/1590570742 learned_addr learned my addr 172.21.15.22:0/1590570742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0659c640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0813c470 0x7f7d0815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0759e640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0813cc40 0x7f7d0815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 -- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0813cc40 msgr2=0x7f7d0815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0813cc40 0x7f7d0815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 -- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0813c470 msgr2=0x7f7d0815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0813c470 0x7f7d0815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 -- 172.21.15.22:0/1590570742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0813b3d0 con 0x7f7cf0004810 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d06d9d640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000fc0 tx=0x7f7cf8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8010040 con 0x7f7cf0004810 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf801a070 con 0x7f7cf0004810 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0813b6b0 con 0x7f7cf0004810 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800fd20 con 0x7f7cf0004810 2024-04-27T07:51:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0813bc70 con 0x7f7cf0004810 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d08068c30 con 0x7f7cf0004810 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7cf801f460 con 0x7f7cf0004810 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 0x7f7cf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf8089a70 con 0x7f7cf0004810 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0659c640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 0x7f7cf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:32.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.751+0000 7f7d0659c640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 0x7f7cf406a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00057f0 tx=0x7f7cfc0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:32.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.755+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7cf805a030 con 0x7f7cf0004810 2024-04-27T07:51:32.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.931+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d0813c050 con 0x7f7cf0004810 2024-04-27T07:51:32.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.931+0000 7f7cef7fe640 1 -- 172.21.15.22:0/1590570742 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f7cf8059930 con 0x7f7cf0004810 2024-04-27T07:51:32.938 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 msgr2=0x7f7cf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 0x7f7cf406a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00057f0 tx=0x7f7cfc0011a0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 msgr2=0x7f7d0813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000fc0 tx=0x7f7cf8011d20 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 shutdown_connections 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7cf4068380 0x7f7cf406a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0813cc40 0x7f7d0815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0813c470 0x7f7d0815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 --2- 172.21.15.22:0/1590570742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf0004810 0x7f7d0813adf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 >> 172.21.15.22:0/1590570742 conn(0x7f7d08067f10 msgr2=0x7f7d0806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 shutdown_connections 2024-04-27T07:51:32.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:32.935+0000 7f7d0d0c8640 1 -- 172.21.15.22:0/1590570742 wait complete. 2024-04-27T07:51:32.957 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:32.957 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:32.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:33.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 Processor -- start 2024-04-27T07:51:33.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 -- start start 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d0151cf0 0x7f72d00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f72d0072fd0 0x7f72d006e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d01520d0 con 0x7f72d00b5740 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f72d00b3590 con 0x7f72d0072fd0 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.079+0000 7f72d494f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f72d00b3870 con 0x7f72d0151cf0 2024-04-27T07:51:33.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f72d0072fd0 0x7f72d006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f72d0072fd0 0x7f72d006e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57066/0 (socket says 172.21.15.22:57066) 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 -- 172.21.15.22:0/4048717686 learned_addr learned my addr 172.21.15.22:0/4048717686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ce59c640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d0151cf0 0x7f72d00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 -- 172.21.15.22:0/4048717686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d0151cf0 msgr2=0x7f72d00b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d0151cf0 0x7f72d00b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 -- 172.21.15.22:0/4048717686 >> v1:172.21.15.183:6789/0 conn(0x7f72d00b5740 legacy=0x7f72d00728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 -- 172.21.15.22:0/4048717686 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d00b3b50 con 0x7f72d0072fd0 2024-04-27T07:51:33.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72ced9d640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0072fd0 0x7f72d006e150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d180 tx=0x7f72c801eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d282fe5862083781 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72cdd9b640 1 -- 172.21.15.22:0/4048717686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c800b2d0 con 0x7f72d0072fd0 2024-04-27T07:51:33.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72cdd9b640 1 -- 172.21.15.22:0/4048717686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c800b470 con 0x7f72d0072fd0 2024-04-27T07:51:33.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72cdd9b640 1 -- 172.21.15.22:0/4048717686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c80a7bc0 con 0x7f72d0072fd0 2024-04-27T07:51:33.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 -- 172.21.15.22:0/4048717686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0072fd0 msgr2=0x7f72d006e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0072fd0 0x7f72d006e150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d180 tx=0x7f72c801eb50 comp rx=0 tx=0).stop 2024-04-27T07:51:33.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 -- 172.21.15.22:0/4048717686 shutdown_connections 2024-04-27T07:51:33.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0072fd0 0x7f72d006e150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 --2- 172.21.15.22:0/4048717686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d0151cf0 0x7f72d00b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 -- 172.21.15.22:0/4048717686 >> 172.21.15.22:0/4048717686 conn(0x7f72d0067f10 msgr2=0x7f72d006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 -- 172.21.15.22:0/4048717686 shutdown_connections 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.083+0000 7f72d494f640 1 -- 172.21.15.22:0/4048717686 wait complete. 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 Processor -- start 2024-04-27T07:51:33.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- start start 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 0x7f72d0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d00b5740 0x7f72d015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 0x7f72d0147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72d006e810 con 0x7f72d0151cf0 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72d006e690 con 0x7f72d0072fd0 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f72d006e990 con 0x7f72d00b5740 2024-04-27T07:51:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ce59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 0x7f72d0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 0x7f72d0147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ce59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 0x7f72d0150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:42682/0 (socket says 172.21.15.22:42682) 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ce59c640 1 -- 172.21.15.22:0/3354190774 learned_addr learned my addr 172.21.15.22:0/3354190774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72c7fff640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d00b5740 0x7f72d015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 -- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d00b5740 msgr2=0x7f72d015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d00b5740 0x7f72d015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 -- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 msgr2=0x7f72d0150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 0x7f72d0150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 -- 172.21.15.22:0/3354190774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d0164c70 con 0x7f72d0151cf0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72ced9d640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 0x7f72d0147240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72c80ad000 tx=0x7f72c801b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8009800 con 0x7f72d0151cf0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c80be070 con 0x7f72d0151cf0 2024-04-27T07:51:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8009c00 con 0x7f72d0151cf0 2024-04-27T07:51:33.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d0164f50 con 0x7f72d0151cf0 2024-04-27T07:51:33.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d0150550 con 0x7f72d0151cf0 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.087+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d0150340 con 0x7f72d0151cf0 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.091+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f72c80b6070 con 0x7f72d0151cf0 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.091+0000 7f72c6ffd640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 0x7f72b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.091+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f72c8121800 con 0x7f72d0151cf0 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.091+0000 7f72ce59c640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 0x7f72b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.091+0000 7f72ce59c640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 0x7f72b006a870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007a70 tx=0x7f72c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.095+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72c8018800 con 0x7f72d0151cf0 2024-04-27T07:51:33.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.255+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72d0068c30 con 0x7f72d0151cf0 2024-04-27T07:51:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.259+0000 7f72c6ffd640 1 -- 172.21.15.22:0/3354190774 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f72c80f1760 con 0x7f72d0151cf0 2024-04-27T07:51:33.265 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:33.265 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.259+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 msgr2=0x7f72b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.259+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 0x7f72b006a870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007a70 tx=0x7f72c0008040 comp rx=0 tx=0).stop 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.259+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 msgr2=0x7f72d0147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.259+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 0x7f72d0147240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72c80ad000 tx=0x7f72c801b8f0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 shutdown_connections 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f72b00683b0 0x7f72b006a870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72d0151cf0 0x7f72d0147240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f72d00b5740 0x7f72d015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 --2- 172.21.15.22:0/3354190774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d0072fd0 0x7f72d0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 >> 172.21.15.22:0/3354190774 conn(0x7f72d0067f10 msgr2=0x7f72d00cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 shutdown_connections 2024-04-27T07:51:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.263+0000 7f72d494f640 1 -- 172.21.15.22:0/3354190774 wait complete. 2024-04-27T07:51:33.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:33.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.399+0000 7f3ece06c640 1 Processor -- start 2024-04-27T07:51:33.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 -- start start 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3ec815ecd0 0x7f3ec8072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3ec80cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec80b5390 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec80b5670 con 0x7f3ec815ecd0 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ece06c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80ce340 con 0x7f3ec80cee20 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec6ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3ec80b4ee0 0x7f3ec80b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57548/0 (socket says 172.21.15.22:57548) 2024-04-27T07:51:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec6ffd640 1 -- 172.21.15.22:0/1013104783 learned_addr learned my addr 172.21.15.22:0/1013104783 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1013104783 >> v2:172.21.15.183:3300/0 conn(0x7f3ec815ecd0 0x7f3ec8072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3ec80cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776919681 0 0) 0x7f3ec80b5390 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eb0003530 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933737655 0 0) 0x7f3eb0003530 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 msgr2=0x7f3ec80cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3ec80cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 >> v2:172.21.15.183:3300/0 conn(0x7f3ec815ecd0 msgr2=0x7f3ec8072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 --2- 172.21.15.22:0/1013104783 >> v2:172.21.15.183:3300/0 conn(0x7f3ec815ecd0 0x7f3ec8072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec80ce620 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f3ebc001680 con 0x7f3ec80b4ee0 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1013104783 >> v2:172.21.15.183:3300/0 conn(0x7f3ec815ecd0 0x7f3ec8072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 >> v1:172.21.15.22:6789/0 conn(0x7f3ec80b4ee0 legacy=0x7f3ec80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3eb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3eb00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b5670 con 0x7f3eb0004d50 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80ce340 con 0x7f3eb00039c0 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb000b9d0 con 0x7f3ec80cee20 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3eb00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3eb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 -- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 msgr2=0x7f3eb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3eb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 -- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb00039c0 msgr2=0x7f3eb0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec7fff640 1 -- 172.21.15.22:0/1013104783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb0003530 con 0x7f3eb0004d50 2024-04-27T07:51:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.403+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec80cee20 0x7f3eb0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3eb00051d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009ae0 tx=0x7f3ec001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5098dc618762c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00b2070 con 0x7f3eb0004d50 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec00a5660 con 0x7f3eb0004d50 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ec67fc640 1 -- 172.21.15.22:0/1013104783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00096b0 con 0x7f3eb0004d50 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- 172.21.15.22:0/1013104783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 msgr2=0x7f3eb00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1013104783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3eb00051d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009ae0 tx=0x7f3ec001ca70 comp rx=0 tx=0).stop 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ec7fff640 1 -- 172.21.15.22:0/1013104783 reap_dead start 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- 172.21.15.22:0/1013104783 shutdown_connections 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- 172.21.15.22:0/1013104783 >> 172.21.15.22:0/1013104783 conn(0x7f3ec80b21d0 msgr2=0x7f3ec80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- 172.21.15.22:0/1013104783 shutdown_connections 2024-04-27T07:51:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- 172.21.15.22:0/1013104783 wait complete. 2024-04-27T07:51:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 Processor -- start 2024-04-27T07:51:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- start start 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 0x7f3ec815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec806e570 0x7f3ec806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b3be0 con 0x7f3eb0004d50 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8072ec0 con 0x7f3ec81f46e0 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.407+0000 7f3ece06c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80aaa00 con 0x7f3ec806e570 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57088/0 (socket says 172.21.15.22:57088) 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 -- 172.21.15.22:0/1378608031 learned_addr learned my addr 172.21.15.22:0/1378608031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 0x7f3ec815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec7fff640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec806e570 0x7f3ec806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 -- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec806e570 msgr2=0x7f3ec806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec806e570 0x7f3ec806e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 -- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 msgr2=0x7f3ec815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 0x7f3ec815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 -- 172.21.15.22:0/1378608031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec80a9960 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 0x7f3ec815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec77fe640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8011500 tx=0x7f3eb80119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb801a020 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb8016680 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec80a9c40 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8016a30 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec80aa200 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec804ff10 con 0x7f3eb0004d50 2024-04-27T07:51:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3eb801d020 con 0x7f3eb0004d50 2024-04-27T07:51:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 0x7f3eb406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3eb8087aa0 con 0x7f3eb0004d50 2024-04-27T07:51:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.411+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 0x7f3eb406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.415+0000 7f3ec6ffd640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 0x7f3eb406a870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc001ec0 tx=0x7f3ebc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.415+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3eb8024030 con 0x7f3eb0004d50 2024-04-27T07:51:33.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ec80a9740 con 0x7f3eb0004d50 2024-04-27T07:51:33.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3eaffff640 1 -- 172.21.15.22:0/1378608031 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3eb8058070 con 0x7f3eb0004d50 2024-04-27T07:51:33.584 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:33.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 msgr2=0x7f3eb406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 0x7f3eb406a870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc001ec0 tx=0x7f3ebc009040 comp rx=0 tx=0).stop 2024-04-27T07:51:33.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 msgr2=0x7f3ec814c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8011500 tx=0x7f3eb80119c0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 shutdown_connections 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eb40683b0 0x7f3eb406a870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec806e570 0x7f3ec806e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec81f46e0 0x7f3ec815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 --2- 172.21.15.22:0/1378608031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3eb0004d50 0x7f3ec814c010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 >> 172.21.15.22:0/1378608031 conn(0x7f3ec80b21d0 msgr2=0x7f3ec80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 shutdown_connections 2024-04-27T07:51:33.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.579+0000 7f3ece06c640 1 -- 172.21.15.22:0/1378608031 wait complete. 2024-04-27T07:51:33.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 Processor -- start 2024-04-27T07:51:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 -- start start 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 0x7f455c072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f455c0cffe0 0x7f455c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c072890 con 0x7f455c06ec80 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0ce0d0 con 0x7f455c0cffe0 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f4562c73640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0cd8c0 con 0x7f455c071f70 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f455bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f455c0cffe0 0x7f455c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f455bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f455c0cffe0 0x7f455c06e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57100/0 (socket says 172.21.15.22:57100) 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.711+0000 7f455bfff640 1 -- 172.21.15.22:0/3062169767 learned_addr learned my addr 172.21.15.22:0/3062169767 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455bfff640 1 -- 172.21.15.22:0/3062169767 >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 msgr2=0x7f455c072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4560a0f640 1 --2- 172.21.15.22:0/3062169767 >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 0x7f455c072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455bfff640 1 --2- 172.21.15.22:0/3062169767 >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 0x7f455c072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455bfff640 1 -- 172.21.15.22:0/3062169767 >> v1:172.21.15.183:6789/0 conn(0x7f455c06ec80 legacy=0x7f455c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455bfff640 1 -- 172.21.15.22:0/3062169767 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0cdb10 con 0x7f455c0cffe0 2024-04-27T07:51:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4560a0f640 1 --2- 172.21.15.22:0/3062169767 >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 0x7f455c072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455bfff640 1 --2- 172.21.15.22:0/3062169767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c0cffe0 0x7f455c06e740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f455c06cf00 tx=0x7f454801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e435df6208e5748d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455b7fe640 1 -- 172.21.15.22:0/3062169767 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480b0070 con 0x7f455c0cffe0 2024-04-27T07:51:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455b7fe640 1 -- 172.21.15.22:0/3062169767 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45480ab600 con 0x7f455c0cffe0 2024-04-27T07:51:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f455b7fe640 1 -- 172.21.15.22:0/3062169767 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480ab9d0 con 0x7f455c0cffe0 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 -- 172.21.15.22:0/3062169767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c0cffe0 msgr2=0x7f455c06e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 --2- 172.21.15.22:0/3062169767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c0cffe0 0x7f455c06e740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f455c06cf00 tx=0x7f454801c450 comp rx=0 tx=0).stop 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 -- 172.21.15.22:0/3062169767 shutdown_connections 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 --2- 172.21.15.22:0/3062169767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c0cffe0 0x7f455c06e740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 --2- 172.21.15.22:0/3062169767 >> v2:172.21.15.183:3300/0 conn(0x7f455c071f70 0x7f455c072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 -- 172.21.15.22:0/3062169767 >> 172.21.15.22:0/3062169767 conn(0x7f455c0b21d0 msgr2=0x7f455c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 -- 172.21.15.22:0/3062169767 shutdown_connections 2024-04-27T07:51:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.715+0000 7f4562c73640 1 -- 172.21.15.22:0/3062169767 wait complete. 2024-04-27T07:51:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 Processor -- start 2024-04-27T07:51:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- start start 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f455c06ec80 0x7f455c13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 0x7f455c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f455c073130 con 0x7f455c071f70 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f455c072fb0 con 0x7f455c0cffe0 2024-04-27T07:51:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f455c0cceb0 con 0x7f455c06ec80 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4561210640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 0x7f455c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4560a0f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f455c06ec80 0x7f455c13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4561210640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 0x7f455c0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:42728/0 (socket says 172.21.15.22:42728) 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57112/0 (socket says 172.21.15.22:57112) 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4561210640 1 -- 172.21.15.22:0/3269082670 learned_addr learned my addr 172.21.15.22:0/3269082670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 -- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f455c06ec80 msgr2=0x7f455c13aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f455c06ec80 0x7f455c13aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 -- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 msgr2=0x7f455c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 0x7f455c0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 -- 172.21.15.22:0/3269082670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c13b420 con 0x7f455c071f70 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f455bfff640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f455c073980 tx=0x7f45480191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480b0040 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45480b4440 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1423c0 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480b4810 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c142980 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c04ff10 con 0x7f455c071f70 2024-04-27T07:51:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f45480bb020 con 0x7f455c071f70 2024-04-27T07:51:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 0x7f454006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.719+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f454811fbc0 con 0x7f455c071f70 2024-04-27T07:51:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.723+0000 7f4560a0f640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 0x7f454006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.723+0000 7f4560a0f640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 0x7f454006a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f455000ab60 tx=0x7f4550008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.723+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45480c2020 con 0x7f455c071f70 2024-04-27T07:51:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.879+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f455c0ce250 con 0x7f455c071f70 2024-04-27T07:51:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.879+0000 7f45597fa640 1 -- 172.21.15.22:0/3269082670 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f45480160d0 con 0x7f455c071f70 2024-04-27T07:51:33.885 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.879+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 msgr2=0x7f454006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.879+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 0x7f454006a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f455000ab60 tx=0x7f4550008040 comp rx=0 tx=0).stop 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.879+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 msgr2=0x7f455c0b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f455c073980 tx=0x7f45480191f0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 shutdown_connections 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4540068380 0x7f454006a840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f455c0cffe0 0x7f455c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f455c071f70 0x7f455c0b45a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 --2- 172.21.15.22:0/3269082670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f455c06ec80 0x7f455c13aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:33.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 >> 172.21.15.22:0/3269082670 conn(0x7f455c0b21d0 msgr2=0x7f455c144ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:33.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 shutdown_connections 2024-04-27T07:51:33.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:33.883+0000 7f4562c73640 1 -- 172.21.15.22:0/3269082670 wait complete. 2024-04-27T07:51:33.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.011+0000 7f3892b53640 1 Processor -- start 2024-04-27T07:51:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.011+0000 7f3892b53640 1 -- start start 2024-04-27T07:51:34.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f388c0b05b0 0x7f388c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f388c06bbd0 con 0x7f388c0d0830 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f388c0cd9f0 con 0x7f388c0b0b00 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f388c06be90 con 0x7f388c0b05b0 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38910f0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f388c0b05b0 0x7f388c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38908ef640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f388c0d0830 0x7f388c0d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57562/0 (socket says 172.21.15.22:57562) 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38908ef640 1 -- 172.21.15.22:0/3826556923 learned_addr learned my addr 172.21.15.22:0/3826556923 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38910f0640 1 -- 172.21.15.22:0/3826556923 >> v1:172.21.15.183:6789/0 conn(0x7f388c0b0b00 legacy=0x7f388c06f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38910f0640 1 -- 172.21.15.22:0/3826556923 >> v1:172.21.15.22:6789/0 conn(0x7f388c0d0830 legacy=0x7f388c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38910f0640 1 -- 172.21.15.22:0/3826556923 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c06c170 con 0x7f388c0b05b0 2024-04-27T07:51:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38837fe640 1 -- 172.21.15.22:0/3826556923 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038527022 0 0) 0x7f388c06bbd0 con 0x7f388c0d0830 2024-04-27T07:51:34.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38910f0640 1 --2- 172.21.15.22:0/3826556923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0b05b0 0x7f388c06f830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3884009d90 tx=0x7f388401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f4bad4625c8839 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38837fe640 1 -- 172.21.15.22:0/3826556923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840b2070 con 0x7f388c0b05b0 2024-04-27T07:51:34.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38837fe640 1 -- 172.21.15.22:0/3826556923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38840a5940 con 0x7f388c0b05b0 2024-04-27T07:51:34.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f38837fe640 1 -- 172.21.15.22:0/3826556923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840a5d10 con 0x7f388c0b05b0 2024-04-27T07:51:34.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- 172.21.15.22:0/3826556923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0b05b0 msgr2=0x7f388c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 --2- 172.21.15.22:0/3826556923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0b05b0 0x7f388c06f830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3884009d90 tx=0x7f388401c970 comp rx=0 tx=0).stop 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- 172.21.15.22:0/3826556923 shutdown_connections 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 --2- 172.21.15.22:0/3826556923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0b05b0 0x7f388c06f830 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- 172.21.15.22:0/3826556923 >> 172.21.15.22:0/3826556923 conn(0x7f388c09aa30 msgr2=0x7f388c0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- 172.21.15.22:0/3826556923 shutdown_connections 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.015+0000 7f3892b53640 1 -- 172.21.15.22:0/3826556923 wait complete. 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 Processor -- start 2024-04-27T07:51:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- start start 2024-04-27T07:51:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 0x7f388c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f388c0b0b00 0x7f388c0cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f388c16cfc0 con 0x7f388c0d0830 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f388c16ce40 con 0x7f388c0b05b0 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f388c16d140 con 0x7f388c0b0b00 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57118/0 (socket says 172.21.15.22:57118) 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 -- 172.21.15.22:0/884361045 learned_addr learned my addr 172.21.15.22:0/884361045 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38908ef640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 0x7f388c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 -- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f388c0b0b00 msgr2=0x7f388c0cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3883fff640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f388c0b0b00 0x7f388c0cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f388c0b0b00 0x7f388c0cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 -- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 msgr2=0x7f388c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 0x7f388c0b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 -- 172.21.15.22:0/884361045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c0b2680 con 0x7f388c0d0830 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38908ef640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 0x7f388c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38910f0640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f38840b0040 tx=0x7f3884019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840b2050 con 0x7f388c0d0830 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38840be070 con 0x7f388c0d0830 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c0b2960 con 0x7f388c0d0830 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840adaf0 con 0x7f388c0d0830 2024-04-27T07:51:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c0b2f20 con 0x7f388c0d0830 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c04ff10 con 0x7f388c0d0830 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.019+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38840adc90 con 0x7f388c0d0830 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.023+0000 7f38817fa640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 0x7f387806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.023+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3884121b30 con 0x7f388c0d0830 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.023+0000 7f38908ef640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 0x7f387806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.023+0000 7f38908ef640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 0x7f387806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3888000f70 tx=0x7f3888009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.027+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3884016560 con 0x7f388c0d0830 2024-04-27T07:51:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.183+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f388c06bbd0 con 0x7f388c0d0830 2024-04-27T07:51:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.183+0000 7f38817fa640 1 -- 172.21.15.22:0/884361045 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f38840ab340 con 0x7f388c0d0830 2024-04-27T07:51:34.189 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 msgr2=0x7f387806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 0x7f387806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3888000f70 tx=0x7f3888009040 comp rx=0 tx=0).stop 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 msgr2=0x7f388c0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f38840b0040 tx=0x7f3884019710 comp rx=0 tx=0).stop 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 shutdown_connections 2024-04-27T07:51:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3878068380 0x7f387806a840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f388c0d0830 0x7f388c0c8830 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f388c0b0b00 0x7f388c0cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 --2- 172.21.15.22:0/884361045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f388c0b05b0 0x7f388c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 >> 172.21.15.22:0/884361045 conn(0x7f388c09aa30 msgr2=0x7f388c15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 shutdown_connections 2024-04-27T07:51:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.187+0000 7f3892b53640 1 -- 172.21.15.22:0/884361045 wait complete. 2024-04-27T07:51:34.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:34.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.315+0000 7fae7ecca640 1 Processor -- start 2024-04-27T07:51:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.315+0000 7fae7ecca640 1 -- start start 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.315+0000 7fae7ecca640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fae7809aa30 0x7fae7809aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae780d0670 con 0x7fae7815c850 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae780cd300 con 0x7fae780d0190 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fae7809bef0 con 0x7fae7809aa30 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7d267640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fae7809aa30 0x7fae7809aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ca66640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fae7815c850 0x7fae7809b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57570/0 (socket says 172.21.15.22:57570) 2024-04-27T07:51:34.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ca66640 1 -- 172.21.15.22:0/3659542333 learned_addr learned my addr 172.21.15.22:0/3659542333 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:34.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7d267640 1 -- 172.21.15.22:0/3659542333 >> v1:172.21.15.183:6789/0 conn(0x7fae780d0190 legacy=0x7fae780cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:34.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7d267640 1 -- 172.21.15.22:0/3659542333 >> v1:172.21.15.22:6789/0 conn(0x7fae7815c850 legacy=0x7fae7809b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7d267640 1 -- 172.21.15.22:0/3659542333 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae780cd5e0 con 0x7fae7809aa30 2024-04-27T07:51:34.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7d267640 1 --2- 172.21.15.22:0/3659542333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae7809aa30 0x7fae7809aee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae700097c0 tx=0x7fae7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d584743c81fb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae777fe640 1 -- 172.21.15.22:0/3659542333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700b0070 con 0x7fae7809aa30 2024-04-27T07:51:34.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae777fe640 1 -- 172.21.15.22:0/3659542333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae700ab600 con 0x7fae7809aa30 2024-04-27T07:51:34.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae777fe640 1 -- 172.21.15.22:0/3659542333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700ab9d0 con 0x7fae7809aa30 2024-04-27T07:51:34.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- 172.21.15.22:0/3659542333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae7809aa30 msgr2=0x7fae7809aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3659542333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae7809aa30 0x7fae7809aee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae700097c0 tx=0x7fae7001c460 comp rx=0 tx=0).stop 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- 172.21.15.22:0/3659542333 shutdown_connections 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3659542333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae7809aa30 0x7fae7809aee0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- 172.21.15.22:0/3659542333 >> 172.21.15.22:0/3659542333 conn(0x7fae780b40b0 msgr2=0x7fae780b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- 172.21.15.22:0/3659542333 shutdown_connections 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.319+0000 7fae7ecca640 1 -- 172.21.15.22:0/3659542333 wait complete. 2024-04-27T07:51:34.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 Processor -- start 2024-04-27T07:51:34.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- start start 2024-04-27T07:51:34.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 0x7fae78150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 0x7fae780c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fae7816a900 con 0x7fae780d0190 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae7816a780 con 0x7fae7815c850 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fae7816aa80 con 0x7fae7809aa30 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57136/0 (socket says 172.21.15.22:57136) 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 -- 172.21.15.22:0/3415373648 learned_addr learned my addr 172.21.15.22:0/3415373648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ca66640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 0x7fae78150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7d267640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 0x7fae780c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 -- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 msgr2=0x7fae78150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 0x7fae78150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 -- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 msgr2=0x7fae780c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 0x7fae780c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 -- 172.21.15.22:0/3415373648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7815f250 con 0x7fae780d0190 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7d267640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 0x7fae780c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ca66640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 0x7fae78150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae77fff640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c450 tx=0x7fae6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c012830 con 0x7fae780d0190 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c017070 con 0x7fae780d0190 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7815f530 con 0x7fae780d0190 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c012c30 con 0x7fae780d0190 2024-04-27T07:51:34.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7815faf0 con 0x7fae780d0190 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7815fd30 con 0x7fae780d0190 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.323+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fae6c01c460 con 0x7fae780d0190 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.327+0000 7fae757fa640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 0x7fae6406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.327+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fae6c086720 con 0x7fae780d0190 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.327+0000 7fae7ca66640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 0x7fae6406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.327+0000 7fae7ca66640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 0x7fae6406a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae78157420 tx=0x7fae68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.331+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae7815fd30 con 0x7fae780d0190 2024-04-27T07:51:34.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.511+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae78069340 con 0x7fae780d0190 2024-04-27T07:51:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.511+0000 7fae757fa640 1 -- 172.21.15.22:0/3415373648 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fae6c056d70 con 0x7fae780d0190 2024-04-27T07:51:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 msgr2=0x7fae6406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 0x7fae6406a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae78157420 tx=0x7fae68009040 comp rx=0 tx=0).stop 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 msgr2=0x7fae780c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c450 tx=0x7fae6c00c910 comp rx=0 tx=0).stop 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 shutdown_connections 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fae64068380 0x7fae6406a840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae7815c850 0x7fae780c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae780d0190 0x7fae780c7f50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 --2- 172.21.15.22:0/3415373648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae7809aa30 0x7fae78150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 >> 172.21.15.22:0/3415373648 conn(0x7fae780b40b0 msgr2=0x7fae78158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 shutdown_connections 2024-04-27T07:51:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.515+0000 7fae7ecca640 1 -- 172.21.15.22:0/3415373648 wait complete. 2024-04-27T07:51:34.535 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi183.front.sepia.ceph.com... 2024-04-27T07:51:34.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:34.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-27T07:51:34.621 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:34.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:51:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:34.622 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-27T07:51:34.637 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:34.637 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:34.645 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:34.645 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:34.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:34.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 Processor -- start 2024-04-27T07:51:34.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 -- start start 2024-04-27T07:51:34.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0614162da0 0x7f061406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f061406fc70 0x7f06140b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f061406e630 con 0x7f06140ce260 2024-04-27T07:51:34.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0614163180 con 0x7f0614162da0 2024-04-27T07:51:34.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0618f16640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f061406f1b0 con 0x7f061406fc70 2024-04-27T07:51:34.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0614162da0 0x7f061406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0614162da0 0x7f061406e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57138/0 (socket says 172.21.15.22:57138) 2024-04-27T07:51:34.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 -- 172.21.15.22:0/2443223020 learned_addr learned my addr 172.21.15.22:0/2443223020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:34.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f0612d9d640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f061406fc70 0x7f06140b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 -- 172.21.15.22:0/2443223020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f061406fc70 msgr2=0x7f06140b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:34.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f061406fc70 0x7f06140b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 -- 172.21.15.22:0/2443223020 >> v1:172.21.15.183:6789/0 conn(0x7f06140ce260 legacy=0x7f061406f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.879+0000 7f061259c640 1 -- 172.21.15.22:0/2443223020 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061406e910 con 0x7f0614162da0 2024-04-27T07:51:34.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f061259c640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0614162da0 0x7f061406e050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0604009870 tx=0x7f060401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38f94bca0a0dd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f061159a640 1 -- 172.21.15.22:0/2443223020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040b2070 con 0x7f0614162da0 2024-04-27T07:51:34.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f061159a640 1 -- 172.21.15.22:0/2443223020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06040a9070 con 0x7f0614162da0 2024-04-27T07:51:34.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f061159a640 1 -- 172.21.15.22:0/2443223020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040ad880 con 0x7f0614162da0 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- 172.21.15.22:0/2443223020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0614162da0 msgr2=0x7f061406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0614162da0 0x7f061406e050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0604009870 tx=0x7f060401c560 comp rx=0 tx=0).stop 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- 172.21.15.22:0/2443223020 shutdown_connections 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f061406fc70 0x7f06140b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 --2- 172.21.15.22:0/2443223020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0614162da0 0x7f061406e050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- 172.21.15.22:0/2443223020 >> 172.21.15.22:0/2443223020 conn(0x7f0614067f10 msgr2=0x7f061406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- 172.21.15.22:0/2443223020 shutdown_connections 2024-04-27T07:51:34.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- 172.21.15.22:0/2443223020 wait complete. 2024-04-27T07:51:34.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 Processor -- start 2024-04-27T07:51:34.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.883+0000 7f0618f16640 1 -- start start 2024-04-27T07:51:34.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06140ce260 0x7f0614154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 0x7f06140b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06140b5700 con 0x7f061406fc70 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06140b5580 con 0x7f0614162da0 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f06140b5880 con 0x7f06140ce260 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57142/0 (socket says 172.21.15.22:57142) 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 -- 172.21.15.22:0/4197484737 learned_addr learned my addr 172.21.15.22:0/4197484737 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0612d9d640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 0x7f06140b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0611d9b640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06140ce260 0x7f0614154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 -- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06140ce260 msgr2=0x7f0614154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06140ce260 0x7f0614154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 -- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 msgr2=0x7f06140b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 0x7f06140b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 -- 172.21.15.22:0/4197484737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0614155a00 con 0x7f061406fc70 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0612d9d640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 0x7f06140b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f061259c640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f06040b0040 tx=0x7f0604019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040b2040 con 0x7f061406fc70 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06040bb070 con 0x7f061406fc70 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040b6440 con 0x7f061406fc70 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0614157f80 con 0x7f061406fc70 2024-04-27T07:51:34.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061415d0b0 con 0x7f061406fc70 2024-04-27T07:51:34.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0614068c30 con 0x7f061406fc70 2024-04-27T07:51:34.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f06040ab070 con 0x7f061406fc70 2024-04-27T07:51:34.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 0x7f05f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:34.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f06041203b0 con 0x7f061406fc70 2024-04-27T07:51:34.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.887+0000 7f0611d9b640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 0x7f05f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:34.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.891+0000 7f0611d9b640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 0x7f05f406a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0608007fd0 tx=0x7f0608009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:34.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:34.891+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0604016150 con 0x7f061406fc70 2024-04-27T07:51:35.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.051+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f061415c590 con 0x7f061406fc70 2024-04-27T07:51:35.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.051+0000 7f0602ffd640 1 -- 172.21.15.22:0/4197484737 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f06040f0a00 con 0x7f061406fc70 2024-04-27T07:51:35.059 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:35.059 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:50:49.784244+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:35.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 msgr2=0x7f05f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 0x7f05f406a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0608007fd0 tx=0x7f0608009040 comp rx=0 tx=0).stop 2024-04-27T07:51:35.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 msgr2=0x7f0614157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f06040b0040 tx=0x7f0604019300 comp rx=0 tx=0).stop 2024-04-27T07:51:35.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 shutdown_connections 2024-04-27T07:51:35.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f05f4068380 0x7f05f406a840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0614162da0 0x7f06140b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06140ce260 0x7f0614154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 --2- 172.21.15.22:0/4197484737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061406fc70 0x7f0614157a40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 >> 172.21.15.22:0/4197484737 conn(0x7f0614067f10 msgr2=0x7f06140ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 shutdown_connections 2024-04-27T07:51:35.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.055+0000 7f0618f16640 1 -- 172.21.15.22:0/4197484737 wait complete. 2024-04-27T07:51:35.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:35.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 Processor -- start 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 -- start start 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95bc167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc0cdb10 con 0x7f95bc0cffe0 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc0cddf0 con 0x7f95bc071f70 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95c0eab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc0ce0d0 con 0x7f95bc06f750 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b9d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f95bc0cffe0 0x7f95bc164390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57582/0 (socket says 172.21.15.22:57582) 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b9d9b640 1 -- 172.21.15.22:0/3637686097 learned_addr learned my addr 172.21.15.22:0/3637686097 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572004371 0 0) 0x7f95bc0cdb10 con 0x7f95bc0cffe0 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95bc167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a8003530 con 0x7f95bc0cffe0 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3512876255 0 0) 0x7f95a8003530 con 0x7f95bc0cffe0 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 msgr2=0x7f95bc167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95bc167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 >> v1:172.21.15.183:6789/0 conn(0x7f95bc071f70 legacy=0x7f95bc072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc06e220 con 0x7f95bc0cffe0 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f95a4001680 con 0x7f95bc0cffe0 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 >> v1:172.21.15.22:6789/0 conn(0x7f95bc0cffe0 legacy=0x7f95bc164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95a80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc0ce0d0 con 0x7f95a8004d50 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95a800bb60 con 0x7f95a8003f70 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95a800bd60 con 0x7f95bc06f750 2024-04-27T07:51:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95ba59c640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95a80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95b9d9b640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 -- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 msgr2=0x7f95a80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95a80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 -- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a8003f70 msgr2=0x7f95a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95bad9d640 1 -- 172.21.15.22:0/3637686097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a8003530 con 0x7f95a8004d50 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.191+0000 7f95ba59c640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc06f750 0x7f95a80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009f70 tx=0x7f95ac01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb26e50d6077e1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0b2070 con 0x7f95a8004d50 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac0a5af0 con 0x7f95a8004d50 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95b959a640 1 -- 172.21.15.22:0/3637686097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac009bf0 con 0x7f95a8004d50 2024-04-27T07:51:35.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- 172.21.15.22:0/3637686097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a8004d50 msgr2=0x7f95a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 --2- 172.21.15.22:0/3637686097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009f70 tx=0x7f95ac01cc20 comp rx=0 tx=0).stop 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 -- 172.21.15.22:0/3637686097 reap_dead start 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- 172.21.15.22:0/3637686097 shutdown_connections 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- 172.21.15.22:0/3637686097 >> 172.21.15.22:0/3637686097 conn(0x7f95bc0b21d0 msgr2=0x7f95bc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- 172.21.15.22:0/3637686097 shutdown_connections 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- 172.21.15.22:0/3637686097 wait complete. 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 Processor -- start 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- start start 2024-04-27T07:51:35.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95bc1400f0 0x7f95bc0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 0x7f95bc140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc06dfd0 con 0x7f95bc1412f0 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc0cd8c0 con 0x7f95bc1400f0 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95c0eab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc072fb0 con 0x7f95bc0b53c0 2024-04-27T07:51:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57164/0 (socket says 172.21.15.22:57164) 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 -- 172.21.15.22:0/167803050 learned_addr learned my addr 172.21.15.22:0/167803050 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95ba59c640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95bc1400f0 0x7f95bc0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95b9d9b640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 0x7f95bc140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.195+0000 7f95bad9d640 1 -- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 msgr2=0x7f95bc140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95bad9d640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 0x7f95bc140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95bad9d640 1 -- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95bc1400f0 msgr2=0x7f95bc0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95bad9d640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95bc1400f0 0x7f95bc0c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95bad9d640 1 -- 172.21.15.22:0/167803050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0ca1a0 con 0x7f95bc1412f0 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b9d9b640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 0x7f95bc140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95bad9d640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0b0040 tx=0x7f95ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0b2040 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac0ad940 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc0ca480 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac009bf0 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc0caa40 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc0cb220 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95ac0c0030 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 0x7f959c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95ac121190 con 0x7f95bc1412f0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95ba59c640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 0x7f959c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.199+0000 7f95ba59c640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 0x7f959c06a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c10 tx=0x7f95b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.203+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95ac0c7060 con 0x7f95bc1412f0 2024-04-27T07:51:35.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.363+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95bc04ff10 con 0x7f95bc1412f0 2024-04-27T07:51:35.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95b2ffd640 1 -- 172.21.15.22:0/167803050 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f95ac016930 con 0x7f95bc1412f0 2024-04-27T07:51:35.371 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 msgr2=0x7f959c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 0x7f959c06a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c10 tx=0x7f95b4009040 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 msgr2=0x7f95bc0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0b0040 tx=0x7f95ac009750 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 shutdown_connections 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f959c068380 0x7f959c06a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95bc1412f0 0x7f95bc0b35d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95bc0b53c0 0x7f95bc140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 --2- 172.21.15.22:0/167803050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95bc1400f0 0x7f95bc0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 >> 172.21.15.22:0/167803050 conn(0x7f95bc0b21d0 msgr2=0x7f95bc142ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 shutdown_connections 2024-04-27T07:51:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.367+0000 7f95c0eab640 1 -- 172.21.15.22:0/167803050 wait complete. 2024-04-27T07:51:35.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:35.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.499+0000 7fc2d2454640 1 Processor -- start 2024-04-27T07:51:35.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2d2454640 1 -- start start 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2d2454640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 0x7fc2cc06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2d2454640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc15c2a0 con 0x7fc2cc15af30 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2d2454640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc15b4e0 con 0x7fc2cc0af990 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2d2454640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc15b7c0 con 0x7fc2cc072f10 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cbfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc2cc15af30 0x7fc2cc072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57584/0 (socket says 172.21.15.22:57584) 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cbfff640 1 -- 172.21.15.22:0/3223339137 learned_addr learned my addr 172.21.15.22:0/3223339137 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 0x7fc2cc06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172563087 0 0) 0x7fc2cc15c2a0 con 0x7fc2cc15af30 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2b4003530 con 0x7fc2cc15af30 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978882992 0 0) 0x7fc2cc15b4e0 con 0x7fc2cc0af990 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2cc15c2a0 con 0x7fc2cc0af990 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2012181667 0 0) 0x7fc2b4003530 con 0x7fc2cc15af30 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 msgr2=0x7fc2cc06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 0x7fc2cc06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 >> v1:172.21.15.183:6789/0 conn(0x7fc2cc0af990 legacy=0x7fc2cc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc15baa0 con 0x7fc2cc15af30 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fc2bc001680 con 0x7fc2cc15af30 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 >> v1:172.21.15.22:6789/0 conn(0x7fc2cc15af30 legacy=0x7fc2cc072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2b4003b80 0x7fc2b4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 0x7fc2b40051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc15b7c0 con 0x7fc2b4004810 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b400bae0 con 0x7fc2b4003b80 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b400def0 con 0x7fc2cc072f10 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cbfff640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2b4003b80 0x7fc2b4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 -- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 msgr2=0x7fc2b40051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc072f10 0x7fc2b40051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 -- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2b4003b80 msgr2=0x7fc2b4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2b4003b80 0x7fc2b4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cb7fe640 1 -- 172.21.15.22:0/3223339137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b4003530 con 0x7fc2b4004810 2024-04-27T07:51:35.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.503+0000 7fc2cbfff640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2b4003b80 0x7fc2b4003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2b4004c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000aa60 tx=0x7fc2c001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c26dc70062e12b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00b2070 con 0x7fc2b4004810 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c00a5a10 con 0x7fc2b4004810 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2caffd640 1 -- 172.21.15.22:0/3223339137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000b040 con 0x7fc2b4004810 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- 172.21.15.22:0/3223339137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 msgr2=0x7fc2b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 --2- 172.21.15.22:0/3223339137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2b4004c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000aa60 tx=0x7fc2c001cc70 comp rx=0 tx=0).stop 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d09f1640 1 -- 172.21.15.22:0/3223339137 reap_dead start 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- 172.21.15.22:0/3223339137 shutdown_connections 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- 172.21.15.22:0/3223339137 >> 172.21.15.22:0/3223339137 conn(0x7fc2cc0b21d0 msgr2=0x7fc2cc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- 172.21.15.22:0/3223339137 shutdown_connections 2024-04-27T07:51:35.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- 172.21.15.22:0/3223339137 wait complete. 2024-04-27T07:51:35.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 Processor -- start 2024-04-27T07:51:35.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- start start 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc153020 0x7fc2cc15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 0x7fc2cc15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b3fb0 con 0x7fc2b4004810 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b3e30 con 0x7fc2cc157de0 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d2454640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b4130 con 0x7fc2cc153020 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57176/0 (socket says 172.21.15.22:57176) 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2cbfff640 1 -- 172.21.15.22:0/1766660756 learned_addr learned my addr 172.21.15.22:0/1766660756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2d09f1640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 0x7fc2cc15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.507+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc153020 0x7fc2cc15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 -- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc153020 msgr2=0x7fc2cc15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc153020 0x7fc2cc15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 -- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 msgr2=0x7fc2cc15fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 0x7fc2cc15fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 -- 172.21.15.22:0/1766660756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc158310 con 0x7fc2b4004810 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2d09f1640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 0x7fc2cc15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cbfff640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00d7a0 tx=0x7fc2bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc019070 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0074e0 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1585f0 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc004a10 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc158bb0 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc04ff10 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2bc01c070 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 0x7fc2b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc2bc0867a0 con 0x7fc2b4004810 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 0x7fc2b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.511+0000 7fc2cb7fe640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 0x7fc2b806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00af000 tx=0x7fc2c00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.515+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2bc023020 con 0x7fc2b4004810 2024-04-27T07:51:35.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc2cc15e0e0 con 0x7fc2b4004810 2024-04-27T07:51:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2c8ff9640 1 -- 172.21.15.22:0/1766660756 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc2bc056e40 con 0x7fc2b4004810 2024-04-27T07:51:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 msgr2=0x7fc2b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 0x7fc2b806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00af000 tx=0x7fc2c00046d0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 msgr2=0x7fc2cc1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00d7a0 tx=0x7fc2bc00dc60 comp rx=0 tx=0).stop 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 shutdown_connections 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc2b80683b0 0x7fc2b806a870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2cc157de0 0x7fc2cc15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2cc153020 0x7fc2cc15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.679+0000 7fc2d2454640 1 --2- 172.21.15.22:0/1766660756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4004810 0x7fc2cc1578a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.683+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 >> 172.21.15.22:0/1766660756 conn(0x7fc2cc0b21d0 msgr2=0x7fc2cc15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.683+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 shutdown_connections 2024-04-27T07:51:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.683+0000 7fc2d2454640 1 -- 172.21.15.22:0/1766660756 wait complete. 2024-04-27T07:51:35.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.823+0000 7fd2647f7640 1 Processor -- start 2024-04-27T07:51:35.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.823+0000 7fd2647f7640 1 -- start start 2024-04-27T07:51:35.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd2647f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd25c163130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd2647f7640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25c0d5680 con 0x7fd25c0d1940 2024-04-27T07:51:35.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd2647f7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25c0cf4d0 con 0x7fd25c161fa0 2024-04-27T07:51:35.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd2647f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0cfe00 con 0x7fd25c0d0290 2024-04-27T07:51:35.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd25c0d1940 0x7fd25c0b7670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57588/0 (socket says 172.21.15.22:57588) 2024-04-27T07:51:35.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 -- 172.21.15.22:0/1004042354 learned_addr learned my addr 172.21.15.22:0/1004042354 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:35.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd262d94640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd25c163130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884404383 0 0) 0x7fd25c0d5680 con 0x7fd25c0d1940 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd248003530 con 0x7fd25c0d1940 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2465567777 0 0) 0x7fd248003530 con 0x7fd25c0d1940 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 msgr2=0x7fd25c163130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd25c163130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 >> v1:172.21.15.183:6789/0 conn(0x7fd25c161fa0 legacy=0x7fd25c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c0cf7b0 con 0x7fd25c0d1940 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fd258001800 con 0x7fd25c0d1940 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 >> v1:172.21.15.22:6789/0 conn(0x7fd25c0d1940 legacy=0x7fd25c0b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2480039f0 0x7fd248003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd248004810 0x7fd248004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd248005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0cfe00 con 0x7fd248004810 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd24800bb00 con 0x7fd2480039f0 2024-04-27T07:51:35.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd24800bd00 con 0x7fd25c0d0290 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd248004810 0x7fd248004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd262593640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2480039f0 0x7fd248003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd262d94640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd248005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 -- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 msgr2=0x7fd248005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd248005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 -- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2480039f0 msgr2=0x7fd248003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2480039f0 0x7fd248003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd261d92640 1 -- 172.21.15.22:0/1004042354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd248003530 con 0x7fd248004810 2024-04-27T07:51:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.827+0000 7fd262d94640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c0d0290 0x7fd248005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd248004810 0x7fd248004c90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c210 tx=0x7fd25801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3800d22e9f69ce server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258007d30 con 0x7fd248004810 2024-04-27T07:51:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2580a9070 con 0x7fd248004810 2024-04-27T07:51:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261591640 1 -- 172.21.15.22:0/1004042354 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580074d0 con 0x7fd248004810 2024-04-27T07:51:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- 172.21.15.22:0/1004042354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd248004810 msgr2=0x7fd248004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 --2- 172.21.15.22:0/1004042354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd248004810 0x7fd248004c90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c210 tx=0x7fd25801cca0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd262d94640 1 -- 172.21.15.22:0/1004042354 reap_dead start 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- 172.21.15.22:0/1004042354 shutdown_connections 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- 172.21.15.22:0/1004042354 >> 172.21.15.22:0/1004042354 conn(0x7fd25c08bfc0 msgr2=0x7fd25c0d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- 172.21.15.22:0/1004042354 shutdown_connections 2024-04-27T07:51:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- 172.21.15.22:0/1004042354 wait complete. 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 Processor -- start 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- start start 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248004810 0x7fd25c0d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c15e0e0 0x7fd25c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0b7d80 con 0x7fd25c0d2be0 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0d5340 con 0x7fd248004810 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd2647f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0b7f00 con 0x7fd25c15e0e0 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd262593640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248004810 0x7fd25c0d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57192/0 (socket says 172.21.15.22:57192) 2024-04-27T07:51:35.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 -- 172.21.15.22:0/219885919 learned_addr learned my addr 172.21.15.22:0/219885919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd262d94640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c15e0e0 0x7fd25c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 -- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c15e0e0 msgr2=0x7fd25c15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c15e0e0 0x7fd25c15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 -- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248004810 msgr2=0x7fd25c0d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248004810 0x7fd25c0d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 -- 172.21.15.22:0/219885919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c1fbca0 con 0x7fd25c0d2be0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.831+0000 7fd261d92640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd25801cad0 tx=0x7fd248005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258000f80 con 0x7fd25c0d2be0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2580074b0 con 0x7fd25c0d2be0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c1fbf80 con 0x7fd25c0d2be0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258007280 con 0x7fd25c0d2be0 2024-04-27T07:51:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c158ef0 con 0x7fd25c0d2be0 2024-04-27T07:51:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c15eaa0 con 0x7fd25c0d2be0 2024-04-27T07:51:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd2580a5870 con 0x7fd25c0d2be0 2024-04-27T07:51:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 0x7fd24006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd258120000 con 0x7fd25c0d2be0 2024-04-27T07:51:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd262593640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 0x7fd24006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.835+0000 7fd262593640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 0x7fd24006a870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd24c001660 tx=0x7fd24c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.839+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2580bf060 con 0x7fd25c0d2be0 2024-04-27T07:51:36.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:35.999+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd25c04ff10 con 0x7fd25c0d2be0 2024-04-27T07:51:36.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd252ffd640 1 -- 172.21.15.22:0/219885919 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fd2580169b0 con 0x7fd25c0d2be0 2024-04-27T07:51:36.008 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 msgr2=0x7fd24006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 0x7fd24006a870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd24c001660 tx=0x7fd24c009040 comp rx=0 tx=0).stop 2024-04-27T07:51:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 msgr2=0x7fd25c158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd25801cad0 tx=0x7fd248005840 comp rx=0 tx=0).stop 2024-04-27T07:51:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 shutdown_connections 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2400683b0 0x7fd24006a870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd25c15e0e0 0x7fd25c15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd25c0d2be0 0x7fd25c158410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 --2- 172.21.15.22:0/219885919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248004810 0x7fd25c0d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 >> 172.21.15.22:0/219885919 conn(0x7fd25c08bfc0 msgr2=0x7fd25c159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 shutdown_connections 2024-04-27T07:51:36.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.003+0000 7fd2647f7640 1 -- 172.21.15.22:0/219885919 wait complete. 2024-04-27T07:51:36.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 Processor -- start 2024-04-27T07:51:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 -- start start 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff22415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22406dba0 con 0x7ff2240cd000 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff224160480 con 0x7ff22406c2e0 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22c3fc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff224161090 con 0x7ff22415c710 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff2240cd000 0x7ff22406bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57592/0 (socket says 172.21.15.22:57592) 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 -- 172.21.15.22:0/1922528879 learned_addr learned my addr 172.21.15.22:0/1922528879 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22a198640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff22415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045569574 0 0) 0x7ff22406dba0 con 0x7ff2240cd000 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff210003530 con 0x7ff2240cd000 2024-04-27T07:51:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 269243665 0 0) 0x7ff210003530 con 0x7ff2240cd000 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 msgr2=0x7ff22415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff22415caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 >> v1:172.21.15.183:6789/0 conn(0x7ff22406c2e0 legacy=0x7ff2240d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff224160760 con 0x7ff2240cd000 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7ff214001680 con 0x7ff2240cd000 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 >> v1:172.21.15.22:6789/0 conn(0x7ff2240cd000 legacy=0x7ff22406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff2100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff210003f70 0x7ff210004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff224161090 con 0x7ff210003f70 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff21000bab0 con 0x7ff210004d50 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff21000bcb0 con 0x7ff22415c710 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff210003f70 0x7ff210004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22a198640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff2100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff22a999640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 -- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 msgr2=0x7ff2100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff22415c710 0x7ff2100039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 -- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff210004d50 msgr2=0x7ff2100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff210004d50 0x7ff2100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.151+0000 7ff229997640 1 -- 172.21.15.22:0/1922528879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff210003530 con 0x7ff210003f70 2024-04-27T07:51:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff229997640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff210003f70 0x7ff210004810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c210 tx=0x7ff21401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1b30af9dc89099 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214007d40 con 0x7ff210003f70 2024-04-27T07:51:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff214005b00 con 0x7ff210003f70 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff229196640 1 -- 172.21.15.22:0/1922528879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214007070 con 0x7ff210003f70 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1922528879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff210003f70 msgr2=0x7ff210004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1922528879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff210003f70 0x7ff210004810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c210 tx=0x7ff21401f3f0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22a999640 1 -- 172.21.15.22:0/1922528879 reap_dead start 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1922528879 shutdown_connections 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1922528879 >> 172.21.15.22:0/1922528879 conn(0x7ff2240b3020 msgr2=0x7ff2240b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1922528879 shutdown_connections 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1922528879 wait complete. 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 Processor -- start 2024-04-27T07:51:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.155+0000 7ff22c3fc640 1 -- start start 2024-04-27T07:51:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff210003f70 0x7ff2240c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2240ca6a0 0x7ff2240cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240b3b40 con 0x7ff22406f160 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff224160d20 con 0x7ff2240ca6a0 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff2240b3cc0 con 0x7ff210003f70 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57200/0 (socket says 172.21.15.22:57200) 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 -- 172.21.15.22:0/1102391685 learned_addr learned my addr 172.21.15.22:0/1102391685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22a999640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2240ca6a0 0x7ff2240cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22a198640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff210003f70 0x7ff2240c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 -- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff210003f70 msgr2=0x7ff2240c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff210003f70 0x7ff2240c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 -- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2240ca6a0 msgr2=0x7ff2240cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2240ca6a0 0x7ff2240cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 -- 172.21.15.22:0/1102391685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2240cb060 con 0x7ff22406f160 2024-04-27T07:51:36.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff229997640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff21401f220 tx=0x7ff210005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140047f0 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2140b6040 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff224070ae0 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214004b90 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff224070000 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff224069340 con 0x7ff22406f160 2024-04-27T07:51:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff214007070 con 0x7ff22406f160 2024-04-27T07:51:36.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 0x7ff20806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff214123c90 con 0x7ff22406f160 2024-04-27T07:51:36.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.159+0000 7ff22a198640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 0x7ff20806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.163+0000 7ff22a198640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 0x7ff20806a870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff21c007520 tx=0x7ff21c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.167+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff214019100 con 0x7ff22406f160 2024-04-27T07:51:36.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2240cbca0 con 0x7ff22406f160 2024-04-27T07:51:36.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff21affd640 1 -- 172.21.15.22:0/1102391685 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7ff2140f4250 con 0x7ff22406f160 2024-04-27T07:51:36.344 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:36.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 msgr2=0x7ff20806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 0x7ff20806a870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff21c007520 tx=0x7ff21c00b040 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 msgr2=0x7ff2240ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff21401f220 tx=0x7ff210005840 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 shutdown_connections 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff2080683b0 0x7ff20806a870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2240ca6a0 0x7ff2240cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff22406f160 0x7ff2240ca160 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 --2- 172.21.15.22:0/1102391685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff210003f70 0x7ff2240c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 >> 172.21.15.22:0/1102391685 conn(0x7ff2240b3020 msgr2=0x7ff2240ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 shutdown_connections 2024-04-27T07:51:36.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.339+0000 7ff22c3fc640 1 -- 172.21.15.22:0/1102391685 wait complete. 2024-04-27T07:51:36.362 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:51:36.363 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:36.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.523+0000 7f3aa27f8640 1 Processor -- start 2024-04-27T07:51:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.523+0000 7f3aa27f8640 1 -- start start 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3aa27f8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3a9c072740 0x7f3a9c072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3aa27f8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3a9c145a00 0x7f3a9c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3aa27f8640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a9c06e050 con 0x7f3a9c1454b0 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3aa27f8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3a9c06f330 con 0x7f3a9c072740 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3aa27f8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3a9c0ceae0 con 0x7f3a9c145a00 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3a9c072740 0x7f3a9c072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3a9c072740 0x7f3a9c072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57214/0 (socket says 172.21.15.22:57214) 2024-04-27T07:51:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 -- 172.21.15.22:0/2600591876 learned_addr learned my addr 172.21.15.22:0/2600591876 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 -- 172.21.15.22:0/2600591876 >> v2:172.21.15.183:3300/0 conn(0x7f3a9c145a00 msgr2=0x7f3a9c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 --2- 172.21.15.22:0/2600591876 >> v2:172.21.15.183:3300/0 conn(0x7f3a9c145a00 0x7f3a9c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 -- 172.21.15.22:0/2600591876 >> v1:172.21.15.22:6789/0 conn(0x7f3a9c1454b0 legacy=0x7f3a9c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 -- 172.21.15.22:0/2600591876 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c0cedc0 con 0x7f3a9c072740 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9b7fe640 1 --2- 172.21.15.22:0/2600591876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c072fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b3510 tx=0x7f3a8401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b114d096d96ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9affd640 1 -- 172.21.15.22:0/2600591876 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a56c0 con 0x7f3a9c072740 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9affd640 1 -- 172.21.15.22:0/2600591876 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a840a5860 con 0x7f3a9c072740 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.527+0000 7f3a9affd640 1 -- 172.21.15.22:0/2600591876 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a5c30 con 0x7f3a9c072740 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/2600591876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 msgr2=0x7f3a9c072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/2600591876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c072fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b3510 tx=0x7f3a8401c9f0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/2600591876 shutdown_connections 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/2600591876 >> v2:172.21.15.183:3300/0 conn(0x7f3a9c145a00 0x7f3a9c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/2600591876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c072fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/2600591876 >> 172.21.15.22:0/2600591876 conn(0x7f3a9c067f10 msgr2=0x7f3a9c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/2600591876 shutdown_connections 2024-04-27T07:51:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/2600591876 wait complete. 2024-04-27T07:51:36.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 Processor -- start 2024-04-27T07:51:36.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- start start 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a9c1454b0 0x7f3a9c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a9c145a00 0x7f3a9c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c15d690 con 0x7f3a9c072740 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c15d510 con 0x7f3a9c145a00 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9c15d810 con 0x7f3a9c1454b0 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57222/0 (socket says 172.21.15.22:57222) 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 -- 172.21.15.22:0/3831736582 learned_addr learned my addr 172.21.15.22:0/3831736582 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa0d95640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a9c145a00 0x7f3a9c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9b7fe640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a9c1454b0 0x7f3a9c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 -- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a9c1454b0 msgr2=0x7f3a9c13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a9c1454b0 0x7f3a9c13e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 -- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a9c145a00 msgr2=0x7f3a9c13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a9c145a00 0x7f3a9c13e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 -- 172.21.15.22:0/3831736582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c15c0e0 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a9bfff640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c450 tx=0x7f3a9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94016020 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a940052d0 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c15bb20 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007260 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c0b12a0 con 0x7f3a9c072740 2024-04-27T07:51:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.531+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9c068c30 con 0x7f3a9c072740 2024-04-27T07:51:36.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.535+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a94007400 con 0x7f3a9c072740 2024-04-27T07:51:36.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.535+0000 7f3a98ff9640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 0x7f3a7c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.535+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3a94086370 con 0x7f3a9c072740 2024-04-27T07:51:36.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.535+0000 7f3a9b7fe640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 0x7f3a7c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.535+0000 7f3a9b7fe640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 0x7f3a7c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a840ae000 tx=0x7f3a840a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.539+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a94020020 con 0x7f3a9c072740 2024-04-27T07:51:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.695+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a9c06f640 con 0x7f3a9c072740 2024-04-27T07:51:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.695+0000 7f3a98ff9640 1 -- 172.21.15.22:0/3831736582 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16774 (secure 0 0 0) 0x7f3a94005480 con 0x7f3a9c072740 2024-04-27T07:51:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:36.703 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":22,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:36.366481+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 msgr2=0x7f3a7c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 0x7f3a7c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a840ae000 tx=0x7f3a840a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 msgr2=0x7f3a9c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c450 tx=0x7f3a9400c910 comp rx=0 tx=0).stop 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 shutdown_connections 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3a7c068380 0x7f3a7c06a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a9c145a00 0x7f3a9c13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a9c1454b0 0x7f3a9c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 --2- 172.21.15.22:0/3831736582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9c072740 0x7f3a9c15a5c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 >> 172.21.15.22:0/3831736582 conn(0x7f3a9c067f10 msgr2=0x7f3a9c140040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 shutdown_connections 2024-04-27T07:51:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.699+0000 7f3aa27f8640 1 -- 172.21.15.22:0/3831736582 wait complete. 2024-04-27T07:51:36.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:36.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 Processor -- start 2024-04-27T07:51:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 -- start start 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f69e409aa30 0x7f69e40b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69e409ae10 0x7f69e409b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e40cea80 con 0x7f69e40d0510 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f69e40ced60 con 0x7f69e409aa30 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e8e00640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f69e40b3700 con 0x7f69e409ae10 2024-04-27T07:51:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f69e409aa30 0x7f69e40b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f69e409aa30 0x7f69e40b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57234/0 (socket says 172.21.15.22:57234) 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 -- 172.21.15.22:0/1835145806 learned_addr learned my addr 172.21.15.22:0/1835145806 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 -- 172.21.15.22:0/1835145806 >> v2:172.21.15.183:3300/0 conn(0x7f69e409ae10 msgr2=0x7f69e409b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 --2- 172.21.15.22:0/1835145806 >> v2:172.21.15.183:3300/0 conn(0x7f69e409ae10 0x7f69e409b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 -- 172.21.15.22:0/1835145806 >> v1:172.21.15.183:6789/0 conn(0x7f69e40d0510 legacy=0x7f69e40d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.847+0000 7f69e259c640 1 -- 172.21.15.22:0/1835145806 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40cdeb0 con 0x7f69e409aa30 2024-04-27T07:51:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e259c640 1 --2- 172.21.15.22:0/1835145806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409aa30 0x7f69e40b39e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69e40d25a0 tx=0x7f69d401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcf99c905b82388 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e159a640 1 -- 172.21.15.22:0/1835145806 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d40b0070 con 0x7f69e409aa30 2024-04-27T07:51:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e159a640 1 -- 172.21.15.22:0/1835145806 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d40ab600 con 0x7f69e409aa30 2024-04-27T07:51:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e159a640 1 -- 172.21.15.22:0/1835145806 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d40ab9d0 con 0x7f69e409aa30 2024-04-27T07:51:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- 172.21.15.22:0/1835145806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409aa30 msgr2=0x7f69e40b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- 172.21.15.22:0/1835145806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409aa30 0x7f69e40b39e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69e40d25a0 tx=0x7f69d401c3c0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- 172.21.15.22:0/1835145806 shutdown_connections 2024-04-27T07:51:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- 172.21.15.22:0/1835145806 >> v2:172.21.15.183:3300/0 conn(0x7f69e409ae10 0x7f69e409b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- 172.21.15.22:0/1835145806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409aa30 0x7f69e40b39e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- 172.21.15.22:0/1835145806 >> 172.21.15.22:0/1835145806 conn(0x7f69e40b4100 msgr2=0x7f69e40b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- 172.21.15.22:0/1835145806 shutdown_connections 2024-04-27T07:51:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- 172.21.15.22:0/1835145806 wait complete. 2024-04-27T07:51:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 Processor -- start 2024-04-27T07:51:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- start start 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69e409aa30 0x7f69e40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69e40d0510 0x7f69e415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.851+0000 7f69e8e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40ce550 con 0x7f69e409ae10 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e8e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40ce3d0 con 0x7f69e409aa30 2024-04-27T07:51:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e8e00640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f69e4169df0 con 0x7f69e40d0510 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57236/0 (socket says 172.21.15.22:57236) 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 -- 172.21.15.22:0/3186569412 learned_addr learned my addr 172.21.15.22:0/3186569412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e259c640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69e409aa30 0x7f69e40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e2d9d640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69e40d0510 0x7f69e415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 -- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69e40d0510 msgr2=0x7f69e415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69e40d0510 0x7f69e415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 -- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69e409aa30 msgr2=0x7f69e40ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69e409aa30 0x7f69e40ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 -- 172.21.15.22:0/3186569412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e4158fb0 con 0x7f69e409ae10 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e1d9b640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b550 tx=0x7f69cc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc014070 con 0x7f69e409ae10 2024-04-27T07:51:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc010890 con 0x7f69e409ae10 2024-04-27T07:51:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e415b9d0 con 0x7f69e409ae10 2024-04-27T07:51:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc010c80 con 0x7f69e409ae10 2024-04-27T07:51:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e415bf90 con 0x7f69e409ae10 2024-04-27T07:51:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e404ff10 con 0x7f69e409ae10 2024-04-27T07:51:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69cc017070 con 0x7f69e409ae10 2024-04-27T07:51:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 0x7f69c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f69cc0867d0 con 0x7f69e409ae10 2024-04-27T07:51:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e259c640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 0x7f69c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.855+0000 7f69e259c640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 0x7f69c406a840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69d40ad000 tx=0x7f69d40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:36.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:36.863+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69cc056da0 con 0x7f69e409ae10 2024-04-27T07:51:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.023+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69e409b2a0 con 0x7f69e409ae10 2024-04-27T07:51:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69daffd640 1 -- 172.21.15.22:0/3186569412 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f69cc010e20 con 0x7f69e409ae10 2024-04-27T07:51:37.031 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 msgr2=0x7f69c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 0x7f69c406a840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69d40ad000 tx=0x7f69d40a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 msgr2=0x7f69e4158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b550 tx=0x7f69cc00ba10 comp rx=0 tx=0).stop 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 shutdown_connections 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f69c4068380 0x7f69c406a840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69e40d0510 0x7f69e415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e409ae10 0x7f69e4158a70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 --2- 172.21.15.22:0/3186569412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69e409aa30 0x7f69e40ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 >> 172.21.15.22:0/3186569412 conn(0x7f69e40b4100 msgr2=0x7f69e4166e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 shutdown_connections 2024-04-27T07:51:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.027+0000 7f69e8e00640 1 -- 172.21.15.22:0/3186569412 wait complete. 2024-04-27T07:51:37.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:37.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 Processor -- start 2024-04-27T07:51:37.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 -- start start 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f78071f70 0x7f4f78072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4f780cffe0 0x7f4f7806e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4f7806ec80 0x7f4f7806f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f4f780ce0d0 con 0x7f4f7806ec80 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4f780cd8c0 con 0x7f4f780cffe0 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f7dd30640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7806dfd0 con 0x7f4f78071f70 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4f7806ec80 0x7f4f7806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4f7806ec80 0x7f4f7806f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57238/0 (socket says 172.21.15.22:57238) 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 -- 172.21.15.22:0/1778442994 learned_addr learned my addr 172.21.15.22:0/1778442994 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f777fe640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f78071f70 0x7f4f78072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f6effd640 1 --2- 172.21.15.22:0/1778442994 >> v2:172.21.15.183:3300/0 conn(0x7f4f780cffe0 0x7f4f7806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 -- 172.21.15.22:0/1778442994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f78071f70 msgr2=0x7f4f78072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f78071f70 0x7f4f78072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 -- 172.21.15.22:0/1778442994 >> v2:172.21.15.183:3300/0 conn(0x7f4f780cffe0 msgr2=0x7f4f7806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 --2- 172.21.15.22:0/1778442994 >> v2:172.21.15.183:3300/0 conn(0x7f4f780cffe0 0x7f4f7806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.167+0000 7f4f77fff640 1 -- 172.21.15.22:0/1778442994 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f78072890 con 0x7f4f7806ec80 2024-04-27T07:51:37.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f77fff640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f7806f110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4f7806df20 tx=0x7f4f7001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d44bcf682bdf28 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f76ffd640 1 -- 172.21.15.22:0/1778442994 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f700a55e0 con 0x7f4f7806ec80 2024-04-27T07:51:37.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f76ffd640 1 -- 172.21.15.22:0/1778442994 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f700a5780 con 0x7f4f7806ec80 2024-04-27T07:51:37.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f76ffd640 1 -- 172.21.15.22:0/1778442994 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f700a5b50 con 0x7f4f7806ec80 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/1778442994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 msgr2=0x7f4f7806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f7806f110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4f7806df20 tx=0x7f4f7001c960 comp rx=0 tx=0).stop 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/1778442994 shutdown_connections 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f7806f110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/1778442994 >> v2:172.21.15.183:3300/0 conn(0x7f4f780cffe0 0x7f4f7806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/1778442994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f78071f70 0x7f4f78072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/1778442994 >> 172.21.15.22:0/1778442994 conn(0x7f4f780b21d0 msgr2=0x7f4f780b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/1778442994 shutdown_connections 2024-04-27T07:51:37.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/1778442994 wait complete. 2024-04-27T07:51:37.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 Processor -- start 2024-04-27T07:51:37.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- start start 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f78071f70 0x7f4f781f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 0x7f4f7815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78072fb0 con 0x7f4f7806ec80 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78146470 con 0x7f4f78071f70 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f78073130 con 0x7f4f780cffe0 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57248/0 (socket says 172.21.15.22:57248) 2024-04-27T07:51:37.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 -- 172.21.15.22:0/3717861617 learned_addr learned my addr 172.21.15.22:0/3717861617 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f6effd640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f78071f70 0x7f4f781f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f77fff640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 0x7f4f7815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 -- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 msgr2=0x7f4f7815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 0x7f4f7815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 -- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f78071f70 msgr2=0x7f4f781f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f78071f70 0x7f4f781f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 -- 172.21.15.22:0/3717861617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f781f7580 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f77fff640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 0x7f4f7815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f777fe640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d830 tx=0x7f4f6800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68018020 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f680096a0 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f780ade80 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68009a50 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.171+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f780ae440 con 0x7f4f7806ec80 2024-04-27T07:51:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7804ff10 con 0x7f4f7806ec80 2024-04-27T07:51:37.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f6801b020 con 0x7f4f7806ec80 2024-04-27T07:51:37.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f74ff9640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 0x7f4f5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4f68087160 con 0x7f4f7806ec80 2024-04-27T07:51:37.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f6effd640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 0x7f4f5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.175+0000 7f4f6effd640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 0x7f4f5806a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f780b1ad0 tx=0x7f4f60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.179+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f68022020 con 0x7f4f7806ec80 2024-04-27T07:51:37.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f78072cb0 con 0x7f4f7806ec80 2024-04-27T07:51:37.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f74ff9640 1 -- 172.21.15.22:0/3717861617 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f4f781f7580 con 0x7f4f7806ec80 2024-04-27T07:51:37.344 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:37.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 msgr2=0x7f4f5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 0x7f4f5806a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f780b1ad0 tx=0x7f4f60008040 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 msgr2=0x7f4f780ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d830 tx=0x7f4f6800dcf0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 shutdown_connections 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f58068380 0x7f4f5806a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f780cffe0 0x7f4f7815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f78071f70 0x7f4f781f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 --2- 172.21.15.22:0/3717861617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f7806ec80 0x7f4f780ad8a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 >> 172.21.15.22:0/3717861617 conn(0x7f4f780b21d0 msgr2=0x7f4f78143b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 shutdown_connections 2024-04-27T07:51:37.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.339+0000 7f4f7dd30640 1 -- 172.21.15.22:0/3717861617 wait complete. 2024-04-27T07:51:37.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 Processor -- start 2024-04-27T07:51:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 -- start start 2024-04-27T07:51:37.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9d7c0b0b00 0x7f9d7c0cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7c070130 con 0x7f9d7c0b05b0 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9d7c0d1150 con 0x7f9d7c0d0830 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d843ed640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f9d7c06bbd0 con 0x7f9d7c0b0b00 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57252/0 (socket says 172.21.15.22:57252) 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 -- 172.21.15.22:0/3154365787 learned_addr learned my addr 172.21.15.22:0/3154365787 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d81988640 1 --2- 172.21.15.22:0/3154365787 >> v2:172.21.15.183:3300/0 conn(0x7f9d7c0b0b00 0x7f9d7c0cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 -- 172.21.15.22:0/3154365787 >> v2:172.21.15.183:3300/0 conn(0x7f9d7c0b0b00 msgr2=0x7f9d7c0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 --2- 172.21.15.22:0/3154365787 >> v2:172.21.15.183:3300/0 conn(0x7f9d7c0b0b00 0x7f9d7c0cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 -- 172.21.15.22:0/3154365787 >> v1:172.21.15.183:6789/0 conn(0x7f9d7c0b05b0 legacy=0x7f9d7c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.475+0000 7f9d82189640 1 -- 172.21.15.22:0/3154365787 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c0cd9f0 con 0x7f9d7c0d0830 2024-04-27T07:51:37.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d82189640 1 --2- 172.21.15.22:0/3154365787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c068a20 tx=0x7f9d7801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460366a6fa846fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d81187640 1 -- 172.21.15.22:0/3154365787 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d780a5870 con 0x7f9d7c0d0830 2024-04-27T07:51:37.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d81187640 1 -- 172.21.15.22:0/3154365787 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d780a5a10 con 0x7f9d7c0d0830 2024-04-27T07:51:37.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d81187640 1 -- 172.21.15.22:0/3154365787 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d780a5de0 con 0x7f9d7c0d0830 2024-04-27T07:51:37.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 -- 172.21.15.22:0/3154365787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 msgr2=0x7f9d7c0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 --2- 172.21.15.22:0/3154365787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c068a20 tx=0x7f9d7801c9b0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 -- 172.21.15.22:0/3154365787 shutdown_connections 2024-04-27T07:51:37.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 --2- 172.21.15.22:0/3154365787 >> v2:172.21.15.183:3300/0 conn(0x7f9d7c0b0b00 0x7f9d7c0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 --2- 172.21.15.22:0/3154365787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c0d0c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 -- 172.21.15.22:0/3154365787 >> 172.21.15.22:0/3154365787 conn(0x7f9d7c09aa30 msgr2=0x7f9d7c0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 -- 172.21.15.22:0/3154365787 shutdown_connections 2024-04-27T07:51:37.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.479+0000 7f9d843ed640 1 -- 172.21.15.22:0/3154365787 wait complete. 2024-04-27T07:51:37.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 Processor -- start 2024-04-27T07:51:37.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- start start 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d7c0b05b0 0x7f9d7c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d7c0b0b00 0x7f9d7c0c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c0cd650 con 0x7f9d7c0d0830 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c0cbb50 con 0x7f9d7c0b0b00 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7c0cd7d0 con 0x7f9d7c0b05b0 2024-04-27T07:51:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57256/0 (socket says 172.21.15.22:57256) 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 -- 172.21.15.22:0/2720890550 learned_addr learned my addr 172.21.15.22:0/2720890550 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d81988640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d7c0b0b00 0x7f9d7c0c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d82189640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d7c0b05b0 0x7f9d7c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 -- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d7c0b05b0 msgr2=0x7f9d7c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d7c0b05b0 0x7f9d7c0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 -- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d7c0b0b00 msgr2=0x7f9d7c0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d7c0b0b00 0x7f9d7c0c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 -- 172.21.15.22:0/2720890550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c0b2690 con 0x7f9d7c0d0830 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d8298a640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000b440 tx=0x7f9d7000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70015020 con 0x7f9d7c0d0830 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d70011850 con 0x7f9d7c0d0830 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c0b2970 con 0x7f9d7c0d0830 2024-04-27T07:51:37.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70011c40 con 0x7f9d7c0d0830 2024-04-27T07:51:37.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c0b2f30 con 0x7f9d7c0d0830 2024-04-27T07:51:37.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c04ff10 con 0x7f9d7c0d0830 2024-04-27T07:51:37.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.483+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9d70018020 con 0x7f9d7c0d0830 2024-04-27T07:51:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.487+0000 7f9d6effd640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 0x7f9d6006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.487+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9d700868d0 con 0x7f9d7c0d0830 2024-04-27T07:51:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.487+0000 7f9d82189640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 0x7f9d6006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.487+0000 7f9d82189640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 0x7f9d6006a870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d780ae000 tx=0x7f9d780a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.491+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d7001f020 con 0x7f9d7c0d0830 2024-04-27T07:51:37.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.643+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d7c0cdcf0 con 0x7f9d7c0d0830 2024-04-27T07:51:37.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.643+0000 7f9d6effd640 1 -- 172.21.15.22:0/2720890550 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9d700172c0 con 0x7f9d7c0d0830 2024-04-27T07:51:37.650 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 msgr2=0x7f9d6006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 0x7f9d6006a870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d780ae000 tx=0x7f9d780a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 msgr2=0x7f9d7c15a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000b440 tx=0x7f9d7000b900 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 shutdown_connections 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9d600683b0 0x7f9d6006a870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d7c0d0830 0x7f9d7c15a9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d7c0b0b00 0x7f9d7c0c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 --2- 172.21.15.22:0/2720890550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d7c0b05b0 0x7f9d7c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 >> 172.21.15.22:0/2720890550 conn(0x7f9d7c09aa30 msgr2=0x7f9d7c15be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 shutdown_connections 2024-04-27T07:51:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.647+0000 7f9d843ed640 1 -- 172.21.15.22:0/2720890550 wait complete. 2024-04-27T07:51:37.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 Processor -- start 2024-04-27T07:51:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 -- start start 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a015ae90 0x7fa5a00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 0x7fa5a006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5a006d120 0x7fa5a015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a006d580 con 0x7fa5a006d120 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00aed00 con 0x7fa5a015ea70 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa5a5ac4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a0073c00 con 0x7fa5a015ae90 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5a006d120 0x7fa5a015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5a006d120 0x7fa5a015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57260/0 (socket says 172.21.15.22:57260) 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 -- 172.21.15.22:0/2378764168 learned_addr learned my addr 172.21.15.22:0/2378764168 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59effd640 1 --2- 172.21.15.22:0/2378764168 >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 0x7fa5a006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59f7fe640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a015ae90 0x7fa5a00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 -- 172.21.15.22:0/2378764168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a015ae90 msgr2=0x7fa5a00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a015ae90 0x7fa5a00ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 -- 172.21.15.22:0/2378764168 >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 msgr2=0x7fa5a006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2378764168 >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 0x7fa5a006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59ffff640 1 -- 172.21.15.22:0/2378764168 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00aef50 con 0x7fa5a006d120 2024-04-27T07:51:37.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.775+0000 7fa59effd640 1 --2- 172.21.15.22:0/2378764168 >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 0x7fa5a006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a006d120 0x7fa5a015b270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5a006e0d0 tx=0x7fa59001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dcc463c2536913 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59e7fc640 1 -- 172.21.15.22:0/2378764168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900ab430 con 0x7fa5a006d120 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59e7fc640 1 -- 172.21.15.22:0/2378764168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900b0070 con 0x7fa5a006d120 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59e7fc640 1 -- 172.21.15.22:0/2378764168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900ab7f0 con 0x7fa5a006d120 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2378764168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a006d120 msgr2=0x7fa5a015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a006d120 0x7fa5a015b270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5a006e0d0 tx=0x7fa59001c4f0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2378764168 shutdown_connections 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a006d120 0x7fa5a015b270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2378764168 >> v2:172.21.15.183:3300/0 conn(0x7fa5a015ea70 0x7fa5a006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2378764168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a015ae90 0x7fa5a00ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2378764168 >> 172.21.15.22:0/2378764168 conn(0x7fa5a0072870 msgr2=0x7fa5a0072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2378764168 shutdown_connections 2024-04-27T07:51:37.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2378764168 wait complete. 2024-04-27T07:51:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 Processor -- start 2024-04-27T07:51:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- start start 2024-04-27T07:51:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 0x7fa5a014a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 0x7fa5a00b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00b4fa0 con 0x7fa5a015ea70 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00afad0 con 0x7fa5a015ae90 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa5a5ac4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00b5120 con 0x7fa5a006d120 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57262/0 (socket says 172.21.15.22:57262) 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.779+0000 7fa59ffff640 1 -- 172.21.15.22:0/2167166714 learned_addr learned my addr 172.21.15.22:0/2167166714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59f7fe640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 0x7fa5a014a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59effd640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 0x7fa5a00b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 -- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 msgr2=0x7fa5a014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 0x7fa5a014a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 -- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 msgr2=0x7fa5a00b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 0x7fa5a00b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 -- 172.21.15.22:0/2167166714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00ae9c0 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59f7fe640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 0x7fa5a014a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59effd640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 0x7fa5a00b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59ffff640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa59001c320 tx=0x7fa590019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b9020 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900b4420 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b47f0 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a014e7d0 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a015c360 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a0069340 con 0x7fa5a015ea70 2024-04-27T07:51:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5900bc020 con 0x7fa5a015ea70 2024-04-27T07:51:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 0x7fa58c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa590120800 con 0x7fa5a015ea70 2024-04-27T07:51:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59f7fe640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 0x7fa58c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.783+0000 7fa59f7fe640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 0x7fa58c06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa598009f70 tx=0x7fa59800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:37.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.787+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5900c3030 con 0x7fa5a015ea70 2024-04-27T07:51:37.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.959+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5a0073460 con 0x7fa5a015ea70 2024-04-27T07:51:37.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa587fff640 1 -- 172.21.15.22:0/2167166714 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa590016110 con 0x7fa5a015ea70 2024-04-27T07:51:37.967 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:37.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 msgr2=0x7fa58c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 0x7fa58c06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa598009f70 tx=0x7fa59800b040 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 msgr2=0x7fa5a00b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa59001c320 tx=0x7fa590019290 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 shutdown_connections 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa58c068380 0x7fa58c06a840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5a015ea70 0x7fa5a00b43d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a015ae90 0x7fa5a00b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 --2- 172.21.15.22:0/2167166714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5a006d120 0x7fa5a014a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 >> 172.21.15.22:0/2167166714 conn(0x7fa5a0072870 msgr2=0x7fa5a006fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 shutdown_connections 2024-04-27T07:51:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:37.963+0000 7fa5a5ac4640 1 -- 172.21.15.22:0/2167166714 wait complete. 2024-04-27T07:51:37.986 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-04-27T07:51:37.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:37.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-27T07:51:37.999 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:37.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:51:37.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:38.000 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-27T07:51:38.048 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:38.048 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:38.054 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:38.054 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:38.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:38.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.223+0000 7ff1c69ca640 1 Processor -- start 2024-04-27T07:51:38.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.223+0000 7ff1c69ca640 1 -- start start 2024-04-27T07:51:38.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c69ca640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff1c01454b0 0x7ff1c0072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c69ca640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 0x7ff1c006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c69ca640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c006e050 con 0x7ff1c0145a00 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c69ca640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c006f330 con 0x7ff1c01454b0 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c69ca640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00ceae0 con 0x7ff1c0072740 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff1c01454b0 0x7ff1c0072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff1c01454b0 0x7ff1c0072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57272/0 (socket says 172.21.15.22:57272) 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 -- 172.21.15.22:0/201527647 learned_addr learned my addr 172.21.15.22:0/201527647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c4f67640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 0x7ff1c006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 -- 172.21.15.22:0/201527647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 msgr2=0x7ff1c006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 0x7ff1c006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 -- 172.21.15.22:0/201527647 >> v1:172.21.15.183:6789/0 conn(0x7ff1c0145a00 legacy=0x7ff1c0072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 -- 172.21.15.22:0/201527647 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00cedc0 con 0x7ff1c01454b0 2024-04-27T07:51:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1c4f67640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 0x7ff1c006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1bffff640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c01454b0 0x7ff1c0072fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e324c691fa41d755 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1beffd640 1 -- 172.21.15.22:0/201527647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00b0070 con 0x7ff1c01454b0 2024-04-27T07:51:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1beffd640 1 -- 172.21.15.22:0/201527647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00ab600 con 0x7ff1c01454b0 2024-04-27T07:51:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.227+0000 7ff1beffd640 1 -- 172.21.15.22:0/201527647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00ab9d0 con 0x7ff1c01454b0 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/201527647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c01454b0 msgr2=0x7ff1c0072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c01454b0 0x7ff1c0072fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b001c520 comp rx=0 tx=0).stop 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/201527647 shutdown_connections 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c0072740 0x7ff1c006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/201527647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c01454b0 0x7ff1c0072fa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/201527647 >> 172.21.15.22:0/201527647 conn(0x7ff1c0067f10 msgr2=0x7ff1c006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/201527647 shutdown_connections 2024-04-27T07:51:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/201527647 wait complete. 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 Processor -- start 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- start start 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c0145a00 0x7ff1c00aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c00aab10 0x7ff1c0140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c015d690 con 0x7ff1c0072740 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c015d510 con 0x7ff1c0145a00 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c015d810 con 0x7ff1c00aab10 2024-04-27T07:51:38.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57286/0 (socket says 172.21.15.22:57286) 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 -- 172.21.15.22:0/3907932188 learned_addr learned my addr 172.21.15.22:0/3907932188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bf7fe640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c0145a00 0x7ff1c00aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c4f67640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c00aab10 0x7ff1c0140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 -- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c00aab10 msgr2=0x7ff1c0140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c00aab10 0x7ff1c0140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 -- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c0145a00 msgr2=0x7ff1c00aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c0145a00 0x7ff1c00aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 -- 172.21.15.22:0/3907932188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c013dce0 con 0x7ff1c0072740 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bffff640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ae040 tx=0x7ff1b00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00ba070 con 0x7ff1c0072740 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00b4420 con 0x7ff1c0072740 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c013dfc0 con 0x7ff1c0072740 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00b0040 con 0x7ff1c0072740 2024-04-27T07:51:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.231+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c013e580 con 0x7ff1c0072740 2024-04-27T07:51:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c0068c30 con 0x7ff1c0072740 2024-04-27T07:51:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff1b00bd070 con 0x7ff1c0072740 2024-04-27T07:51:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1bcff9640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 0x7ff1ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff1b011fb90 con 0x7ff1c0072740 2024-04-27T07:51:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1bf7fe640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 0x7ff1ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.235+0000 7ff1bf7fe640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 0x7ff1ac06a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00b2dc0 tx=0x7ff1b4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.239+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1b0016110 con 0x7ff1c0072740 2024-04-27T07:51:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.399+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c006ef00 con 0x7ff1c0072740 2024-04-27T07:51:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1bcff9640 1 -- 172.21.15.22:0/3907932188 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16774 (secure 0 0 0) 0x7ff1b00efc10 con 0x7ff1c0072740 2024-04-27T07:51:38.408 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:38.409 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:37.374737+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 msgr2=0x7ff1ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 0x7ff1ac06a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00b2dc0 tx=0x7ff1b4009060 comp rx=0 tx=0).stop 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 msgr2=0x7ff1c015bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ae040 tx=0x7ff1b00192c0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 shutdown_connections 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff1ac068380 0x7ff1ac06a840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1c00aab10 0x7ff1c0140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c0145a00 0x7ff1c00aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 --2- 172.21.15.22:0/3907932188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1c0072740 0x7ff1c015bc40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.403+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 >> 172.21.15.22:0/3907932188 conn(0x7ff1c0067f10 msgr2=0x7ff1c01426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:38.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.407+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 shutdown_connections 2024-04-27T07:51:38.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.407+0000 7ff1c69ca640 1 -- 172.21.15.22:0/3907932188 wait complete. 2024-04-27T07:51:38.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:38.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.547+0000 7f7942d31640 1 Processor -- start 2024-04-27T07:51:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.547+0000 7f7942d31640 1 -- start start 2024-04-27T07:51:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7942d31640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f793c15af30 0x7f793c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7942d31640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c15c2a0 con 0x7f793c0af990 2024-04-27T07:51:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7942d31640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c15b4e0 con 0x7f793c072f10 2024-04-27T07:51:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7942d31640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f793c15b7c0 con 0x7f793c15af30 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7940acd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f793c15af30 0x7f793c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f793c0af990 0x7f793c072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57612/0 (socket says 172.21.15.22:57612) 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793bfff640 1 -- 172.21.15.22:0/2467645707 learned_addr learned my addr 172.21.15.22:0/2467645707 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7940acd640 1 -- 172.21.15.22:0/2467645707 >> v1:172.21.15.183:6789/0 conn(0x7f793c072f10 legacy=0x7f793c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7940acd640 1 -- 172.21.15.22:0/2467645707 >> v1:172.21.15.22:6789/0 conn(0x7f793c0af990 legacy=0x7f793c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793b7fe640 1 -- 172.21.15.22:0/2467645707 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295748894 0 0) 0x7f793c15c2a0 con 0x7f793c0af990 2024-04-27T07:51:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7940acd640 1 -- 172.21.15.22:0/2467645707 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c15baa0 con 0x7f793c15af30 2024-04-27T07:51:38.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f7940acd640 1 --2- 172.21.15.22:0/2467645707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c15af30 0x7f793c06f490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f793000a960 tx=0x7f793001d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db109a0fd1a629d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793b7fe640 1 -- 172.21.15.22:0/2467645707 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793000b040 con 0x7f793c15af30 2024-04-27T07:51:38.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793b7fe640 1 -- 172.21.15.22:0/2467645707 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300a9660 con 0x7f793c15af30 2024-04-27T07:51:38.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.551+0000 7f793b7fe640 1 -- 172.21.15.22:0/2467645707 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300a9a30 con 0x7f793c15af30 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2467645707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c15af30 msgr2=0x7f793c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 --2- 172.21.15.22:0/2467645707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c15af30 0x7f793c06f490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f793000a960 tx=0x7f793001d7a0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2467645707 shutdown_connections 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 --2- 172.21.15.22:0/2467645707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c15af30 0x7f793c06f490 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2467645707 >> 172.21.15.22:0/2467645707 conn(0x7f793c0b21d0 msgr2=0x7f793c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2467645707 shutdown_connections 2024-04-27T07:51:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2467645707 wait complete. 2024-04-27T07:51:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 Processor -- start 2024-04-27T07:51:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- start start 2024-04-27T07:51:38.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793c072f10 0x7f793c151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f793c15af30 0x7f793c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f793c168c00 con 0x7f793c0af990 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f793c168a80 con 0x7f793c072f10 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f793c168d80 con 0x7f793c15af30 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57294/0 (socket says 172.21.15.22:57294) 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 -- 172.21.15.22:0/2602080822 learned_addr learned my addr 172.21.15.22:0/2602080822 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7940acd640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793c072f10 0x7f793c151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f79412ce640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f793c15af30 0x7f793c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 -- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f793c15af30 msgr2=0x7f793c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f793c15af30 0x7f793c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 -- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793c072f10 msgr2=0x7f793c151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793c072f10 0x7f793c151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 -- 172.21.15.22:0/2602080822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1539d0 con 0x7f793c0af990 2024-04-27T07:51:38.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f793bfff640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f792800c480 tx=0x7f792800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928018070 con 0x7f793c0af990 2024-04-27T07:51:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7928007500 con 0x7f793c0af990 2024-04-27T07:51:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004a30 con 0x7f793c0af990 2024-04-27T07:51:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c0b0150 con 0x7f793c0af990 2024-04-27T07:51:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.555+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c0b0710 con 0x7f793c0af990 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c04ff10 con 0x7f793c0af990 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f792801b070 con 0x7f793c0af990 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f79397fa640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 0x7f792006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7928085d40 con 0x7f793c0af990 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f7940acd640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 0x7f792006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.559+0000 7f7940acd640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 0x7f792006a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79300b0000 tx=0x7f79300b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.563+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7928056310 con 0x7f793c0af990 2024-04-27T07:51:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.727+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f793c15df20 con 0x7f793c0af990 2024-04-27T07:51:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f79397fa640 1 -- 172.21.15.22:0/2602080822 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f7928055c10 con 0x7f793c0af990 2024-04-27T07:51:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 msgr2=0x7f792006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 0x7f792006a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79300b0000 tx=0x7f79300b3040 comp rx=0 tx=0).stop 2024-04-27T07:51:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 msgr2=0x7f793c152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f792800c480 tx=0x7f792800c940 comp rx=0 tx=0).stop 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 shutdown_connections 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7920068380 0x7f792006a840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f793c15af30 0x7f793c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793c0af990 0x7f793c152f10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 --2- 172.21.15.22:0/2602080822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793c072f10 0x7f793c151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 >> 172.21.15.22:0/2602080822 conn(0x7f793c0b21d0 msgr2=0x7f793c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 shutdown_connections 2024-04-27T07:51:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.731+0000 7f7942d31640 1 -- 172.21.15.22:0/2602080822 wait complete. 2024-04-27T07:51:38.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:38.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 Processor -- start 2024-04-27T07:51:38.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 -- start start 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4e1c15c9c0 0x7f4e1c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c0d1340 0x7f4e1c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1c09aad0 con 0x7f4e1c09bb70 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4e1c09adb0 con 0x7f4e1c15c9c0 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e2447d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c09b090 con 0x7f4e1c0d1340 2024-04-27T07:51:38.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e21a18640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4e1c09bb70 0x7f4e1c0d0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57624/0 (socket says 172.21.15.22:57624) 2024-04-27T07:51:38.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e21a18640 1 -- 172.21.15.22:0/1394861937 learned_addr learned my addr 172.21.15.22:0/1394861937 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:38.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22219640 1 --2- 172.21.15.22:0/1394861937 >> v2:172.21.15.183:3300/0 conn(0x7f4e1c15c9c0 0x7f4e1c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22a1a640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c0d1340 0x7f4e1c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847851499 0 0) 0x7f4e1c09aad0 con 0x7f4e1c09bb70 2024-04-27T07:51:38.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e08003530 con 0x7f4e1c09bb70 2024-04-27T07:51:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22219640 1 -- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c0d1340 msgr2=0x7f4e1c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22219640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c0d1340 0x7f4e1c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22219640 1 -- 172.21.15.22:0/1394861937 >> v1:172.21.15.22:6789/0 conn(0x7f4e1c09bb70 legacy=0x7f4e1c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1910948104 0 0) 0x7f4e08003530 con 0x7f4e1c09bb70 2024-04-27T07:51:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.879+0000 7f4e22219640 1 -- 172.21.15.22:0/1394861937 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c09b370 con 0x7f4e1c15c9c0 2024-04-27T07:51:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e22219640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c15c9c0 0x7f4e1c0cdc50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097c0 tx=0x7f4e1801c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7083bd21f18215 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180b0070 con 0x7f4e1c15c9c0 2024-04-27T07:51:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e180ab400 con 0x7f4e1c15c9c0 2024-04-27T07:51:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e21217640 1 -- 172.21.15.22:0/1394861937 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c001680 con 0x7f4e1c15c9c0 2024-04-27T07:51:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 -- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c15c9c0 msgr2=0x7f4e1c0cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c15c9c0 0x7f4e1c0cdc50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097c0 tx=0x7f4e1801c300 comp rx=0 tx=0).stop 2024-04-27T07:51:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 -- 172.21.15.22:0/1394861937 shutdown_connections 2024-04-27T07:51:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c0d1340 0x7f4e1c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 --2- 172.21.15.22:0/1394861937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c15c9c0 0x7f4e1c0cdc50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 -- 172.21.15.22:0/1394861937 >> 172.21.15.22:0/1394861937 conn(0x7f4e1c0b4140 msgr2=0x7f4e1c0b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:38.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 -- 172.21.15.22:0/1394861937 shutdown_connections 2024-04-27T07:51:38.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 -- 172.21.15.22:0/1394861937 wait complete. 2024-04-27T07:51:38.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.883+0000 7f4e2447d640 1 Processor -- start 2024-04-27T07:51:38.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- start start 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c09bb70 0x7f4e1c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 0x7f4e1c0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b2940 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b27c0 con 0x7f4e1c0d1340 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b2ac0 con 0x7f4e1c09bb70 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57308/0 (socket says 172.21.15.22:57308) 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 -- 172.21.15.22:0/3845802216 learned_addr learned my addr 172.21.15.22:0/3845802216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e21a18640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 0x7f4e1c0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22219640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c09bb70 0x7f4e1c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 -- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c09bb70 msgr2=0x7f4e1c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c09bb70 0x7f4e1c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 -- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 msgr2=0x7f4e1c0c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 0x7f4e1c0c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 -- 172.21.15.22:0/3845802216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c155c00 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e21a18640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 0x7f4e1c0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e22a1a640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400ec10 tx=0x7f4e1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14018020 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1400cd40 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e140146c0 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c0b52c0 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c0b5880 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c04ff10 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4e1401b020 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.887+0000 7f4e12ffd640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 0x7f4e0006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.891+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4e14087920 con 0x7f4e1c0c9da0 2024-04-27T07:51:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.891+0000 7f4e22219640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 0x7f4e0006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.891+0000 7f4e22219640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 0x7f4e0006a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ad000 tx=0x7f4e180a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:38.895+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e14022030 con 0x7f4e1c0c9da0 2024-04-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.055+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e1c0ceeb0 con 0x7f4e1c0c9da0 2024-04-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e12ffd640 1 -- 172.21.15.22:0/3845802216 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f4e14010040 con 0x7f4e1c0c9da0 2024-04-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:39.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 msgr2=0x7f4e0006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 0x7f4e0006a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ad000 tx=0x7f4e180a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 msgr2=0x7f4e1c1585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400ec10 tx=0x7f4e1400c490 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 shutdown_connections 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4e00068380 0x7f4e0006a840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0c9da0 0x7f4e1c1585f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e1c0d1340 0x7f4e1c0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 --2- 172.21.15.22:0/3845802216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e1c09bb70 0x7f4e1c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 >> 172.21.15.22:0/3845802216 conn(0x7f4e1c0b4140 msgr2=0x7f4e1c0709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 shutdown_connections 2024-04-27T07:51:39.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.059+0000 7f4e2447d640 1 -- 172.21.15.22:0/3845802216 wait complete. 2024-04-27T07:51:39.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:39.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 Processor -- start 2024-04-27T07:51:39.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 -- start start 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0740cffe0 0x7fb074164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740cdb10 con 0x7fb074071f70 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740cddf0 con 0x7fb07406f750 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb079e29640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb0740ce0d0 con 0x7fb0740cffe0 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb06bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0740cffe0 0x7fb074164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb0737fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb074071f70 0x7fb074072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57628/0 (socket says 172.21.15.22:57628) 2024-04-27T07:51:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb0737fe640 1 -- 172.21.15.22:0/3976017909 learned_addr learned my addr 172.21.15.22:0/3976017909 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb072ffd640 1 -- 172.21.15.22:0/3976017909 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384940434 0 0) 0x7fb0740cdb10 con 0x7fb074071f70 2024-04-27T07:51:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb072ffd640 1 -- 172.21.15.22:0/3976017909 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb060003530 con 0x7fb074071f70 2024-04-27T07:51:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb06bfff640 1 -- 172.21.15.22:0/3976017909 >> v1:172.21.15.183:6789/0 conn(0x7fb07406f750 legacy=0x7fb0741668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb06bfff640 1 -- 172.21.15.22:0/3976017909 >> v1:172.21.15.22:6789/0 conn(0x7fb074071f70 legacy=0x7fb074072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.199+0000 7fb06bfff640 1 -- 172.21.15.22:0/3976017909 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07406e220 con 0x7fb0740cffe0 2024-04-27T07:51:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb06bfff640 1 --2- 172.21.15.22:0/3976017909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0740cffe0 0x7fb074164390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0097c0 tx=0x7fb05c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a88d780d211bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb072ffd640 1 -- 172.21.15.22:0/3976017909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0b0070 con 0x7fb0740cffe0 2024-04-27T07:51:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb072ffd640 1 -- 172.21.15.22:0/3976017909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05c0ab600 con 0x7fb0740cffe0 2024-04-27T07:51:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb072ffd640 1 -- 172.21.15.22:0/3976017909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0ab9d0 con 0x7fb0740cffe0 2024-04-27T07:51:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- 172.21.15.22:0/3976017909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0740cffe0 msgr2=0x7fb074164390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 --2- 172.21.15.22:0/3976017909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0740cffe0 0x7fb074164390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0097c0 tx=0x7fb05c01c410 comp rx=0 tx=0).stop 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- 172.21.15.22:0/3976017909 shutdown_connections 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 --2- 172.21.15.22:0/3976017909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0740cffe0 0x7fb074164390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- 172.21.15.22:0/3976017909 >> 172.21.15.22:0/3976017909 conn(0x7fb0740b21d0 msgr2=0x7fb0740b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- 172.21.15.22:0/3976017909 shutdown_connections 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- 172.21.15.22:0/3976017909 wait complete. 2024-04-27T07:51:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 Processor -- start 2024-04-27T07:51:39.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- start start 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb074071f70 0x7fb07415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 0x7fb07415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb07416a820 con 0x7fb07406f750 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406dfd0 con 0x7fb0740cffe0 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb079e29640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb07416a9a0 con 0x7fb074071f70 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb0737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb0737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57332/0 (socket says 172.21.15.22:57332) 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb0737fe640 1 -- 172.21.15.22:0/4050224570 learned_addr learned my addr 172.21.15.22:0/4050224570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.203+0000 7fb073fff640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 0x7fb07415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 -- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb074071f70 msgr2=0x7fb07415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb074071f70 0x7fb07415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 -- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 msgr2=0x7fb07415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 0x7fb07415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 -- 172.21.15.22:0/4050224570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07415b340 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb073fff640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 0x7fb07415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb0737fe640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c500 tx=0x7fb06400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064017070 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07415b620 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064005cf0 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064007310 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07415bbe0 con 0x7fb07406f750 2024-04-27T07:51:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07404ff10 con 0x7fb07406f750 2024-04-27T07:51:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb06401a070 con 0x7fb07406f750 2024-04-27T07:51:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 0x7fb05406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb0640862b0 con 0x7fb07406f750 2024-04-27T07:51:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb06bfff640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 0x7fb05406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.207+0000 7fb06bfff640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 0x7fb05406a870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0ad000 tx=0x7fb05c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.211+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb064021020 con 0x7fb07406f750 2024-04-27T07:51:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb074072cb0 con 0x7fb07406f750 2024-04-27T07:51:39.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb070ff9640 1 -- 172.21.15.22:0/4050224570 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fb064056880 con 0x7fb07406f750 2024-04-27T07:51:39.380 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 msgr2=0x7fb05406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 0x7fb05406a870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0ad000 tx=0x7fb05c004740 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 msgr2=0x7fb074159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c500 tx=0x7fb06400c9c0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 shutdown_connections 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0540683b0 0x7fb05406a870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0740cffe0 0x7fb07415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb074071f70 0x7fb07415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 --2- 172.21.15.22:0/4050224570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07406f750 0x7fb074159760 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 >> 172.21.15.22:0/4050224570 conn(0x7fb0740b21d0 msgr2=0x7fb0740b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.375+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 shutdown_connections 2024-04-27T07:51:39.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.379+0000 7fb079e29640 1 -- 172.21.15.22:0/4050224570 wait complete. 2024-04-27T07:51:39.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 Processor -- start 2024-04-27T07:51:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 -- start start 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63f8144e40 0x7f63f806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f8145220 con 0x7f63f806f560 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f8162df0 con 0x7f63f806ef10 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f6400492640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f63f81630d0 con 0x7f63f8144e40 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fe22e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63f8144e40 0x7f63f806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fea2f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f63f806f560 0x7f63f80aec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57634/0 (socket says 172.21.15.22:57634) 2024-04-27T07:51:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fea2f640 1 -- 172.21.15.22:0/2599276426 learned_addr learned my addr 172.21.15.22:0/2599276426 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fe22e640 1 -- 172.21.15.22:0/2599276426 >> v1:172.21.15.183:6789/0 conn(0x7f63f806ef10 legacy=0x7f63f81623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fe22e640 1 -- 172.21.15.22:0/2599276426 >> v1:172.21.15.22:6789/0 conn(0x7f63f806f560 legacy=0x7f63f80aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fd22c640 1 -- 172.21.15.22:0/2599276426 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722409150 0 0) 0x7f63f00014a0 con 0x7f63f806f560 2024-04-27T07:51:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.519+0000 7f63fe22e640 1 -- 172.21.15.22:0/2599276426 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f806e1d0 con 0x7f63f8144e40 2024-04-27T07:51:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f63fe22e640 1 --2- 172.21.15.22:0/2599276426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f8144e40 0x7f63f806e9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63f40097c0 tx=0x7f63f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e972ac92297859cc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f63fd22c640 1 -- 172.21.15.22:0/2599276426 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f40b1070 con 0x7f63f8144e40 2024-04-27T07:51:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f63fd22c640 1 -- 172.21.15.22:0/2599276426 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f40ac410 con 0x7f63f8144e40 2024-04-27T07:51:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f63fd22c640 1 -- 172.21.15.22:0/2599276426 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f40ac7e0 con 0x7f63f8144e40 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 -- 172.21.15.22:0/2599276426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f8144e40 msgr2=0x7f63f806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 --2- 172.21.15.22:0/2599276426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f8144e40 0x7f63f806e9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63f40097c0 tx=0x7f63f401c4a0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 -- 172.21.15.22:0/2599276426 shutdown_connections 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 --2- 172.21.15.22:0/2599276426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f8144e40 0x7f63f806e9d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 -- 172.21.15.22:0/2599276426 >> 172.21.15.22:0/2599276426 conn(0x7f63f8067f10 msgr2=0x7f63f806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 -- 172.21.15.22:0/2599276426 shutdown_connections 2024-04-27T07:51:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.523+0000 7f6400492640 1 -- 172.21.15.22:0/2599276426 wait complete. 2024-04-27T07:51:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 Processor -- start 2024-04-27T07:51:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- start start 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 0x7f63f80b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 0x7f63f80cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8072370 con 0x7f63f815dd70 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63f80721f0 con 0x7f63f806f560 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f63f816cd50 con 0x7f63f8144e40 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57362/0 (socket says 172.21.15.22:57362) 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 -- 172.21.15.22:0/1687130212 learned_addr learned my addr 172.21.15.22:0/1687130212 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fe22e640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 0x7f63f80b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fda2d640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 0x7f63f80cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 -- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 msgr2=0x7f63f80cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 0x7f63f80cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 -- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 msgr2=0x7f63f80b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 0x7f63f80b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 -- 172.21.15.22:0/1687130212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f80b0900 con 0x7f63f815dd70 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fe22e640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 0x7f63f80b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fda2d640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 0x7f63f80cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63fea2f640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c7f0 tx=0x7f63f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0018070 con 0x7f63f815dd70 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f000a640 con 0x7f63f815dd70 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f80b0be0 con 0x7f63f815dd70 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000aa30 con 0x7f63f815dd70 2024-04-27T07:51:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f80b11a0 con 0x7f63f815dd70 2024-04-27T07:51:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f8068c30 con 0x7f63f815dd70 2024-04-27T07:51:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f63f001b070 con 0x7f63f815dd70 2024-04-27T07:51:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.527+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f63f0086000 con 0x7f63f815dd70 2024-04-27T07:51:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.531+0000 7f63fe22e640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.531+0000 7f63fe22e640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 0x7f63dc06a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63f40ae000 tx=0x7f63f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.535+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63f8145220 con 0x7f63f815dd70 2024-04-27T07:51:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.707+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f63f8169da0 con 0x7f63f815dd70 2024-04-27T07:51:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f63eeffd640 1 -- 172.21.15.22:0/1687130212 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f63f00565d0 con 0x7f63f815dd70 2024-04-27T07:51:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 msgr2=0x7f63dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 0x7f63dc06a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63f40ae000 tx=0x7f63f4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 msgr2=0x7f63f81f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c7f0 tx=0x7f63f000ccb0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 shutdown_connections 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f815dd70 0x7f63f81f7ef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63f8144e40 0x7f63f80cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 --2- 172.21.15.22:0/1687130212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63f806f560 0x7f63f80b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 >> 172.21.15.22:0/1687130212 conn(0x7f63f8067f10 msgr2=0x7f63f815ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 shutdown_connections 2024-04-27T07:51:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.711+0000 7f6400492640 1 -- 172.21.15.22:0/1687130212 wait complete. 2024-04-27T07:51:39.735 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:51:39.735 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:51:39.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.863+0000 7fd49d58d640 1 Processor -- start 2024-04-27T07:51:39.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.863+0000 7fd49d58d640 1 -- start start 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.863+0000 7fd49d58d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd4980d4790 0x7fd4980d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4980cff70 con 0x7fd4980b7930 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4980b3d30 con 0x7fd49815ebe0 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd4980d4540 con 0x7fd4980d4790 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd4977fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd4980b7930 0x7fd4980d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57642/0 (socket says 172.21.15.22:57642) 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd4977fe640 1 -- 172.21.15.22:0/3660105308 learned_addr learned my addr 172.21.15.22:0/3660105308 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:39.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd496ffd640 1 --2- 172.21.15.22:0/3660105308 >> v2:172.21.15.22:3300/0 conn(0x7fd4980d4790 0x7fd4980d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd495ffb640 1 -- 172.21.15.22:0/3660105308 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969136975 0 0) 0x7fd4980cff70 con 0x7fd4980b7930 2024-04-27T07:51:39.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd495ffb640 1 -- 172.21.15.22:0/3660105308 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd480003530 con 0x7fd4980b7930 2024-04-27T07:51:39.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd496ffd640 1 -- 172.21.15.22:0/3660105308 >> v1:172.21.15.183:6789/0 conn(0x7fd49815ebe0 legacy=0x7fd4980cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd496ffd640 1 -- 172.21.15.22:0/3660105308 >> v1:172.21.15.22:6789/0 conn(0x7fd4980b7930 legacy=0x7fd4980d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd496ffd640 1 -- 172.21.15.22:0/3660105308 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4980ba7a0 con 0x7fd4980d4790 2024-04-27T07:51:39.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd496ffd640 1 --2- 172.21.15.22:0/3660105308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980d4790 0x7fd4980d4b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097c0 tx=0x7fd48801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef91af5accdc092e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd495ffb640 1 -- 172.21.15.22:0/3660105308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4880ab610 con 0x7fd4980d4790 2024-04-27T07:51:39.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd495ffb640 1 -- 172.21.15.22:0/3660105308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4880b0070 con 0x7fd4980d4790 2024-04-27T07:51:39.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd495ffb640 1 -- 172.21.15.22:0/3660105308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4880ab9d0 con 0x7fd4980d4790 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- 172.21.15.22:0/3660105308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980d4790 msgr2=0x7fd4980d4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3660105308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980d4790 0x7fd4980d4b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097c0 tx=0x7fd48801c410 comp rx=0 tx=0).stop 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- 172.21.15.22:0/3660105308 shutdown_connections 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3660105308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980d4790 0x7fd4980d4b70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- 172.21.15.22:0/3660105308 >> 172.21.15.22:0/3660105308 conn(0x7fd498067d10 msgr2=0x7fd498068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- 172.21.15.22:0/3660105308 shutdown_connections 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.867+0000 7fd49d58d640 1 -- 172.21.15.22:0/3660105308 wait complete. 2024-04-27T07:51:39.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 Processor -- start 2024-04-27T07:51:39.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- start start 2024-04-27T07:51:39.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 0x7fd49814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4980b6bc0 0x7fd4980b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4980ba120 con 0x7fd4980b7930 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4980d4260 con 0x7fd4980b6bc0 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd4980ba2a0 con 0x7fd4980d4790 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57376/0 (socket says 172.21.15.22:57376) 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 -- 172.21.15.22:0/3069626388 learned_addr learned my addr 172.21.15.22:0/3069626388 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd4977fe640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4980b6bc0 0x7fd4980b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd4967fc640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 0x7fd49814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 -- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 msgr2=0x7fd49814dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 0x7fd49814dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 -- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4980b6bc0 msgr2=0x7fd4980b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4980b6bc0 0x7fd4980b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 -- 172.21.15.22:0/3069626388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49814e550 con 0x7fd4980b7930 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd4967fc640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 0x7fd49814dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd496ffd640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097c0 tx=0x7fd48801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488006e20 con 0x7fd4980b7930 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488004020 con 0x7fd4980b7930 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49814e830 con 0x7fd4980b7930 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4880043d0 con 0x7fd4980b7930 2024-04-27T07:51:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49814edf0 con 0x7fd4980b7930 2024-04-27T07:51:39.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.871+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd498068c30 con 0x7fd4980b7930 2024-04-27T07:51:39.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.875+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd488003620 con 0x7fd4980b7930 2024-04-27T07:51:39.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.875+0000 7fd47f7fe640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 0x7fd48406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:39.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.875+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd4881211a0 con 0x7fd4980b7930 2024-04-27T07:51:39.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.875+0000 7fd4967fc640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 0x7fd48406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:39.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.875+0000 7fd4967fc640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 0x7fd48406a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4980b9c10 tx=0x7fd48c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:39.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:39.879+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4880a8050 con 0x7fd4980b7930 2024-04-27T07:51:40.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.035+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4980cb340 con 0x7fd4980b7930 2024-04-27T07:51:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd47f7fe640 1 -- 172.21.15.22:0/3069626388 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16774 (secure 0 0 0) 0x7fd488016090 con 0x7fd4980b7930 2024-04-27T07:51:40.043 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:40.043 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:37.374737+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 msgr2=0x7fd48406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 0x7fd48406a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4980b9c10 tx=0x7fd48c009040 comp rx=0 tx=0).stop 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 msgr2=0x7fd49814f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4880097c0 tx=0x7fd48801ac50 comp rx=0 tx=0).stop 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 shutdown_connections 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd484068380 0x7fd48406a840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4980b6bc0 0x7fd4980b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4980d4790 0x7fd49814dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 --2- 172.21.15.22:0/3069626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4980b7930 0x7fd49814f5f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 >> 172.21.15.22:0/3069626388 conn(0x7fd498067d10 msgr2=0x7fd49815d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 shutdown_connections 2024-04-27T07:51:40.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.039+0000 7fd49d58d640 1 -- 172.21.15.22:0/3069626388 wait complete. 2024-04-27T07:51:40.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:40.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 Processor -- start 2024-04-27T07:51:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 -- start start 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f128c072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f128c0cee20 0x7f128c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c0b5390 con 0x7f128c0b4ee0 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f128c0b5670 con 0x7f128c0cee20 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1290a7f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0ce340 con 0x7f128c15ecd0 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f128c0b4ee0 0x7f128c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58976/0 (socket says 172.21.15.22:58976) 2024-04-27T07:51:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 -- 172.21.15.22:0/1443150216 learned_addr learned my addr 172.21.15.22:0/1443150216 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578588963 0 0) 0x7f128c0b5390 con 0x7f128c0b4ee0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1278005740 con 0x7f128c0b4ee0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f128ad9d640 1 --2- 172.21.15.22:0/1443150216 >> v2:172.21.15.183:3300/0 conn(0x7f128c0cee20 0x7f128c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f128a59c640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f128c072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278213219 0 0) 0x7f1278005740 con 0x7f128c0b4ee0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 msgr2=0x7f128c072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f128c072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 >> v2:172.21.15.183:3300/0 conn(0x7f128c0cee20 msgr2=0x7f128c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 --2- 172.21.15.22:0/1443150216 >> v2:172.21.15.183:3300/0 conn(0x7f128c0cee20 0x7f128c0cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0ce620 con 0x7f128c0b4ee0 2024-04-27T07:51:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f127c001680 con 0x7f128c0b4ee0 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 >> v1:172.21.15.22:6789/0 conn(0x7f128c0b4ee0 legacy=0x7f128c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f1278005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1278006200 0x7f1278006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1278006f60 0x7f12780073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0b5670 con 0x7f1278006200 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0ce340 con 0x7f1278006f60 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f127800dce0 con 0x7f128c15ecd0 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f128ad9d640 1 --2- 172.21.15.22:0/1443150216 >> v2:172.21.15.183:3300/0 conn(0x7f128c0cee20 0x7f128c0cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1278006200 0x7f1278006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f128a59c640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f1278005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f128ad9d640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1278006f60 0x7f12780073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 -- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 msgr2=0x7f1278005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f128c15ecd0 0x7f1278005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 -- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1278006f60 msgr2=0x7f12780073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1278006f60 0x7f12780073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.175+0000 7f1281d9b640 1 -- 172.21.15.22:0/1443150216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1278005740 con 0x7f1278006200 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1281d9b640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1278006200 0x7f1278006a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c210 tx=0x7f127c01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f942843c16e8bf28 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c007d40 con 0x7f1278006200 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c005b00 con 0x7f1278006200 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1289d9b640 1 -- 172.21.15.22:0/1443150216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c007070 con 0x7f1278006200 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- 172.21.15.22:0/1443150216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1278006200 msgr2=0x7f1278006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 --2- 172.21.15.22:0/1443150216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1278006200 0x7f1278006a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c210 tx=0x7f127c01f3d0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f128ad9d640 1 -- 172.21.15.22:0/1443150216 reap_dead start 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- 172.21.15.22:0/1443150216 shutdown_connections 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- 172.21.15.22:0/1443150216 >> 172.21.15.22:0/1443150216 conn(0x7f128c0b21d0 msgr2=0x7f128c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- 172.21.15.22:0/1443150216 shutdown_connections 2024-04-27T07:51:40.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- 172.21.15.22:0/1443150216 wait complete. 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 Processor -- start 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- start start 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278006200 0x7f128c14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f128c15a4c0 0x7f128c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0b3be0 con 0x7f128c158950 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f128c072980 con 0x7f128c15a4c0 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1290a7f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f128c1531a0 con 0x7f1278006200 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1281d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1281d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45060/0 (socket says 172.21.15.22:45060) 2024-04-27T07:51:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f1281d9b640 1 -- 172.21.15.22:0/2049107765 learned_addr learned my addr 172.21.15.22:0/2049107765 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f128a59c640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278006200 0x7f128c14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.179+0000 7f128ad9d640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f128c15a4c0 0x7f128c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 -- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278006200 msgr2=0x7f128c14f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278006200 0x7f128c14f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 -- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f128c15a4c0 msgr2=0x7f128c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f128c15a4c0 0x7f128c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 -- 172.21.15.22:0/2049107765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c15ae80 con 0x7f128c158950 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1281d9b640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f127c09bb20 tx=0x7f1278007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c005cb0 con 0x7f128c158950 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c005e50 con 0x7f128c158950 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c15b0f0 con 0x7f128c158950 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0049f0 con 0x7f128c158950 2024-04-27T07:51:40.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c15b6b0 con 0x7f128c158950 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c04ff10 con 0x7f128c158950 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f127c0ba030 con 0x7f128c158950 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 0x7f126c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f127c123480 con 0x7f128c158950 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f128a59c640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 0x7f126c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.183+0000 7f128a59c640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 0x7f126c06a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f128c156a90 tx=0x7f1274008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.187+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f127c0c1030 con 0x7f128c158950 2024-04-27T07:51:40.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.351+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f128c0cfc60 con 0x7f128c158950 2024-04-27T07:51:40.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.351+0000 7f12837fe640 1 -- 172.21.15.22:0/2049107765 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f127c0190e0 con 0x7f128c158950 2024-04-27T07:51:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:40.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 msgr2=0x7f126c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 0x7f126c06a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f128c156a90 tx=0x7f1274008040 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 msgr2=0x7f128c158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f127c09bb20 tx=0x7f1278007a50 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 shutdown_connections 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f126c068380 0x7f126c06a840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f128c15a4c0 0x7f128c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c158950 0x7f128c158dd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 --2- 172.21.15.22:0/2049107765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278006200 0x7f128c14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 >> 172.21.15.22:0/2049107765 conn(0x7f128c0b21d0 msgr2=0x7f128c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 shutdown_connections 2024-04-27T07:51:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.355+0000 7f1290a7f640 1 -- 172.21.15.22:0/2049107765 wait complete. 2024-04-27T07:51:40.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 Processor -- start 2024-04-27T07:51:40.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 -- start start 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3ec0161f30 0x7f3ec00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec006e070 con 0x7f3ec0071f70 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec0072420 con 0x7f3ec0161f30 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.499+0000 7f3ec70eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec00726c0 con 0x7f3ec00afd00 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec5688640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec4e87640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3ec0161f30 0x7f3ec00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ebffff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3ec0071f70 0x7f3ec00af5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38100/0 (socket says 172.21.15.22:38100) 2024-04-27T07:51:40.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ebffff640 1 -- 172.21.15.22:0/675911661 learned_addr learned my addr 172.21.15.22:0/675911661 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:40.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec4e87640 1 -- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 msgr2=0x7f3ec006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec4e87640 1 -- 172.21.15.22:0/675911661 >> v1:172.21.15.183:6789/0 conn(0x7f3ec0071f70 legacy=0x7f3ec00af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ec4e87640 1 -- 172.21.15.22:0/675911661 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec00729a0 con 0x7f3ec0161f30 2024-04-27T07:51:40.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.503+0000 7f3ebf7fe640 1 -- 172.21.15.22:0/675911661 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848157674 0 0) 0x7f3ec006e070 con 0x7f3ec0071f70 2024-04-27T07:51:40.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0161f30 0x7f3ec00b34d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097c0 tx=0x7f3eb001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848ec903e2253906 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ebf7fe640 1 -- 172.21.15.22:0/675911661 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00b0070 con 0x7f3ec0161f30 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ebf7fe640 1 -- 172.21.15.22:0/675911661 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb00ab600 con 0x7f3ec0161f30 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ebf7fe640 1 -- 172.21.15.22:0/675911661 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00ab9d0 con 0x7f3ec0161f30 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0161f30 msgr2=0x7f3ec00b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0161f30 0x7f3ec00b34d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097c0 tx=0x7f3eb001c410 comp rx=0 tx=0).stop 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- 172.21.15.22:0/675911661 shutdown_connections 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/675911661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0161f30 0x7f3ec00b34d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- 172.21.15.22:0/675911661 >> 172.21.15.22:0/675911661 conn(0x7f3ec00b21d0 msgr2=0x7f3ec00b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- 172.21.15.22:0/675911661 shutdown_connections 2024-04-27T07:51:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- 172.21.15.22:0/675911661 wait complete. 2024-04-27T07:51:40.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 Processor -- start 2024-04-27T07:51:40.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- start start 2024-04-27T07:51:40.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 0x7f3ec015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec0168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec006f6d0 con 0x7f3ec0161f30 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec006f550 con 0x7f3ec0071f70 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec70eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec006f850 con 0x7f3ec00afd00 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45072/0 (socket says 172.21.15.22:45072) 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 -- 172.21.15.22:0/81536238 learned_addr learned my addr 172.21.15.22:0/81536238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 0x7f3ec015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ebffff640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec0168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 -- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 msgr2=0x7f3ec0168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec0168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 -- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 msgr2=0x7f3ec015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 0x7f3ec015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 -- 172.21.15.22:0/81536238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec0169280 con 0x7f3ec0161f30 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 0x7f3ec015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.507+0000 7f3ec5688640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009e30 tx=0x7f3eb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8018020 con 0x7f3ec0161f30 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb8014480 con 0x7f3ec0161f30 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec0158f50 con 0x7f3ec0161f30 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8014870 con 0x7f3ec0161f30 2024-04-27T07:51:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec015e460 con 0x7f3ec0161f30 2024-04-27T07:51:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec004ff10 con 0x7f3ec0161f30 2024-04-27T07:51:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3eb801b020 con 0x7f3ec0161f30 2024-04-27T07:51:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 0x7f3eac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3eb8087870 con 0x7f3ec0161f30 2024-04-27T07:51:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 0x7f3eac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.511+0000 7f3ec4e87640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 0x7f3eac06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00ad000 tx=0x7f3eb00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.515+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3eb8022030 con 0x7f3ec0161f30 2024-04-27T07:51:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.675+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ec00b4840 con 0x7f3ec0161f30 2024-04-27T07:51:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ebd7fa640 1 -- 172.21.15.22:0/81536238 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f3ec0169280 con 0x7f3ec0161f30 2024-04-27T07:51:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 msgr2=0x7f3eac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 0x7f3eac06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00ad000 tx=0x7f3eb00a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 msgr2=0x7f3ec0158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009e30 tx=0x7f3eb800c490 comp rx=0 tx=0).stop 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 shutdown_connections 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3eac068380 0x7f3eac06a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec0161f30 0x7f3ec0158a10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec00afd00 0x7f3ec0168d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 --2- 172.21.15.22:0/81536238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec0071f70 0x7f3ec015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 >> 172.21.15.22:0/81536238 conn(0x7f3ec00b21d0 msgr2=0x7f3ec0155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 shutdown_connections 2024-04-27T07:51:40.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.679+0000 7f3ec70eb640 1 -- 172.21.15.22:0/81536238 wait complete. 2024-04-27T07:51:40.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 Processor -- start 2024-04-27T07:51:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 -- start start 2024-04-27T07:51:40.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f46d815ecd0 0x7f46d8072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46d80b4ee0 0x7f46d80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d80b5390 con 0x7f46d80b3a10 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f46d80b5670 con 0x7f46d815ecd0 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46e0701640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f46d80ce340 con 0x7f46d80b4ee0 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46dec9e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f46d80b3a10 0x7f46d80cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58990/0 (socket says 172.21.15.22:58990) 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46dec9e640 1 -- 172.21.15.22:0/786783877 learned_addr learned my addr 172.21.15.22:0/786783877 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46de49d640 1 --2- 172.21.15.22:0/786783877 >> v2:172.21.15.22:3300/0 conn(0x7f46d815ecd0 0x7f46d8072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46ddc9c640 1 --2- 172.21.15.22:0/786783877 >> v2:172.21.15.183:3300/0 conn(0x7f46d80b4ee0 0x7f46d80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110933695 0 0) 0x7f46d80b5390 con 0x7f46d80b3a10 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46c4003530 con 0x7f46d80b3a10 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46de49d640 1 -- 172.21.15.22:0/786783877 >> v2:172.21.15.183:3300/0 conn(0x7f46d80b4ee0 msgr2=0x7f46d80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46de49d640 1 --2- 172.21.15.22:0/786783877 >> v2:172.21.15.183:3300/0 conn(0x7f46d80b4ee0 0x7f46d80b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46de49d640 1 -- 172.21.15.22:0/786783877 >> v1:172.21.15.22:6789/0 conn(0x7f46d80b3a10 legacy=0x7f46d80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2327143852 0 0) 0x7f46c4003530 con 0x7f46d80b3a10 2024-04-27T07:51:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.823+0000 7f46de49d640 1 -- 172.21.15.22:0/786783877 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d80ce620 con 0x7f46d815ecd0 2024-04-27T07:51:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46de49d640 1 --2- 172.21.15.22:0/786783877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d8072980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f46d8073820 tx=0x7f46d401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7685b104ac3c7c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40b1070 con 0x7f46d815ecd0 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d40ac440 con 0x7f46d815ecd0 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dd49b640 1 -- 172.21.15.22:0/786783877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0001a00 con 0x7f46d815ecd0 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- 172.21.15.22:0/786783877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 msgr2=0x7f46d8072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- 172.21.15.22:0/786783877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d8072980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f46d8073820 tx=0x7f46d401c370 comp rx=0 tx=0).stop 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- 172.21.15.22:0/786783877 shutdown_connections 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- 172.21.15.22:0/786783877 >> v2:172.21.15.183:3300/0 conn(0x7f46d80b4ee0 0x7f46d80b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- 172.21.15.22:0/786783877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d8072980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- 172.21.15.22:0/786783877 >> 172.21.15.22:0/786783877 conn(0x7f46d80b21d0 msgr2=0x7f46d80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- 172.21.15.22:0/786783877 shutdown_connections 2024-04-27T07:51:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- 172.21.15.22:0/786783877 wait complete. 2024-04-27T07:51:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 Processor -- start 2024-04-27T07:51:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- start start 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46d80b3a10 0x7f46d8165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46d80b4ee0 0x7f46d8167f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8166d40 con 0x7f46d815ecd0 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8166ec0 con 0x7f46d80b3a10 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46e0701640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f46d8167040 con 0x7f46d80b4ee0 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45088/0 (socket says 172.21.15.22:45088) 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 -- 172.21.15.22:0/2887958477 learned_addr learned my addr 172.21.15.22:0/2887958477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 -- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46d80b4ee0 msgr2=0x7f46d8167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46d80b4ee0 0x7f46d8167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46de49d640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46d80b3a10 0x7f46d8165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 -- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46d80b3a10 msgr2=0x7f46d8165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.827+0000 7f46dec9e640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46d80b3a10 0x7f46d8165160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46dec9e640 1 -- 172.21.15.22:0/2887958477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81686c0 con 0x7f46d815ecd0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46dec9e640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f46d00018a0 tx=0x7f46d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0012400 con 0x7f46d815ecd0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d00125c0 con 0x7f46d815ecd0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00129b0 con 0x7f46d815ecd0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81689a0 con 0x7f46d815ecd0 2024-04-27T07:51:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d8168f60 con 0x7f46d815ecd0 2024-04-27T07:51:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d804ff10 con 0x7f46d815ecd0 2024-04-27T07:51:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f46d0012b50 con 0x7f46d815ecd0 2024-04-27T07:51:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 0x7f46bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f46d00872f0 con 0x7f46d815ecd0 2024-04-27T07:51:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46de49d640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 0x7f46bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.831+0000 7f46de49d640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 0x7f46bc06a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d40ae000 tx=0x7f46d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:40.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.835+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46d00182b0 con 0x7f46d815ecd0 2024-04-27T07:51:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46d80cfc60 con 0x7f46d815ecd0 2024-04-27T07:51:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46ceffd640 1 -- 172.21.15.22:0/2887958477 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f46d0057830 con 0x7f46d815ecd0 2024-04-27T07:51:41.004 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:41.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 msgr2=0x7f46bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 0x7f46bc06a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d40ae000 tx=0x7f46d4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 msgr2=0x7f46d806eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f46d00018a0 tx=0x7f46d000a430 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 shutdown_connections 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f46bc068380 0x7f46bc06a840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d815ecd0 0x7f46d806eca0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46d80b4ee0 0x7f46d8167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 --2- 172.21.15.22:0/2887958477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46d80b3a10 0x7f46d8165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 >> 172.21.15.22:0/2887958477 conn(0x7f46d80b21d0 msgr2=0x7f46d8158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 shutdown_connections 2024-04-27T07:51:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:40.999+0000 7f46e0701640 1 -- 172.21.15.22:0/2887958477 wait complete. 2024-04-27T07:51:41.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:51:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 Processor -- start 2024-04-27T07:51:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 -- start start 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 0x7f39d409b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d40d0190 0x7f39d40cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f39d40ce320 0x7f39d409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40d0950 con 0x7f39d40ce320 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40d0670 con 0x7f39d415c850 2024-04-27T07:51:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39da400640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f39d40cd300 con 0x7f39d40d0190 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f39d40ce320 0x7f39d409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f39d40ce320 0x7f39d409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45104/0 (socket says 172.21.15.22:45104) 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 -- 172.21.15.22:0/4248198296 learned_addr learned my addr 172.21.15.22:0/4248198296 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d3fff640 1 --2- 172.21.15.22:0/4248198296 >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 0x7f39d409b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d37fe640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d40d0190 0x7f39d40cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 -- 172.21.15.22:0/4248198296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d40d0190 msgr2=0x7f39d40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d40d0190 0x7f39d40cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 -- 172.21.15.22:0/4248198296 >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 msgr2=0x7f39d409b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 --2- 172.21.15.22:0/4248198296 >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 0x7f39d409b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d899d640 1 -- 172.21.15.22:0/4248198296 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40cd5e0 con 0x7f39d40ce320 2024-04-27T07:51:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.143+0000 7f39d3fff640 1 --2- 172.21.15.22:0/4248198296 >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 0x7f39d409b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d899d640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40ce320 0x7f39d409aa30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f39d4073670 tx=0x7f39cc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7da641e87a65d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d2ffd640 1 -- 172.21.15.22:0/4248198296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0a5750 con 0x7f39d40ce320 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d2ffd640 1 -- 172.21.15.22:0/4248198296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0a58f0 con 0x7f39d40ce320 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d2ffd640 1 -- 172.21.15.22:0/4248198296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0a5cc0 con 0x7f39d40ce320 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- 172.21.15.22:0/4248198296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40ce320 msgr2=0x7f39d409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40ce320 0x7f39d409aa30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f39d4073670 tx=0x7f39cc01c9c0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- 172.21.15.22:0/4248198296 shutdown_connections 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40ce320 0x7f39d409aa30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- 172.21.15.22:0/4248198296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d40d0190 0x7f39d40cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- 172.21.15.22:0/4248198296 >> v2:172.21.15.183:3300/0 conn(0x7f39d415c850 0x7f39d409b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- 172.21.15.22:0/4248198296 >> 172.21.15.22:0/4248198296 conn(0x7f39d40b40b0 msgr2=0x7f39d40b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- 172.21.15.22:0/4248198296 shutdown_connections 2024-04-27T07:51:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- 172.21.15.22:0/4248198296 wait complete. 2024-04-27T07:51:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 Processor -- start 2024-04-27T07:51:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- start start 2024-04-27T07:51:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 0x7f39d4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d415c850 0x7f39d4165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41605b0 con 0x7f39d40d0190 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4160430 con 0x7f39d40ce320 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39da400640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f39d4160730 con 0x7f39d415c850 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45124/0 (socket says 172.21.15.22:45124) 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d37fe640 1 -- 172.21.15.22:0/2378952287 learned_addr learned my addr 172.21.15.22:0/2378952287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d3fff640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 0x7f39d4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.147+0000 7f39d899d640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d415c850 0x7f39d4165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 -- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d415c850 msgr2=0x7f39d4165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d415c850 0x7f39d4165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 -- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 msgr2=0x7f39d4157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 0x7f39d4157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 -- 172.21.15.22:0/2378952287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d406ecf0 con 0x7f39d40d0190 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d3fff640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 0x7f39d4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d37fe640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c820 tx=0x7f39c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8015070 con 0x7f39d40d0190 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c8004220 con 0x7f39d40d0190 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d414e2d0 con 0x7f39d40d0190 2024-04-27T07:51:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ce10 con 0x7f39d40d0190 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d4165080 con 0x7f39d40d0190 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4069340 con 0x7f39d40d0190 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f39c8007830 con 0x7f39d40d0190 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 0x7f39c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f39c80867e0 con 0x7f39d40d0190 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d3fff640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 0x7f39c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.151+0000 7f39d3fff640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 0x7f39c006a870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b990 tx=0x7f39c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.155+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39c8056d20 con 0x7f39d40d0190 2024-04-27T07:51:41.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.331+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f39d40b4d10 con 0x7f39d40d0190 2024-04-27T07:51:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.331+0000 7f39d0ff9640 1 -- 172.21.15.22:0/2378952287 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f39c8056620 con 0x7f39d40d0190 2024-04-27T07:51:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-04-27T07:51:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 msgr2=0x7f39c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 0x7f39c006a870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b990 tx=0x7f39c4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 msgr2=0x7f39d414dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c820 tx=0x7f39c800cce0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 shutdown_connections 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f39c00683b0 0x7f39c006a870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39d415c850 0x7f39d4165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d40d0190 0x7f39d414dd90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 --2- 172.21.15.22:0/2378952287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39d40ce320 0x7f39d4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 >> 172.21.15.22:0/2378952287 conn(0x7f39d40b40b0 msgr2=0x7f39d415baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 shutdown_connections 2024-04-27T07:51:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.335+0000 7f39da400640 1 -- 172.21.15.22:0/2378952287 wait complete. 2024-04-27T07:51:41.357 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi183.front.sepia.ceph.com... 2024-04-27T07:51:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:41.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-27T07:51:41.371 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:41.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:51:41.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:51:41.371 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-27T07:51:41.422 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-27T07:51:41.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.535+0000 7fd8f4226640 1 Processor -- start 2024-04-27T07:51:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.535+0000 7fd8f4226640 1 -- start start 2024-04-27T07:51:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f4226640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f4226640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd8ec145a00 0x7fd8ec06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f4226640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec06e050 con 0x7fd8ec1454b0 2024-04-27T07:51:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f4226640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec06f330 con 0x7fd8ec145a00 2024-04-27T07:51:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f4226640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec0ceae0 con 0x7fd8ec072740 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f1fc2640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fd8ec1454b0 0x7fd8ec072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38104/0 (socket says 172.21.15.22:38104) 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f1fc2640 1 -- 172.21.15.22:0/456607588 learned_addr learned my addr 172.21.15.22:0/456607588 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f17c1640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/456607588 >> v2:172.21.15.183:3300/0 conn(0x7fd8ec145a00 0x7fd8ec06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f0fc0640 1 -- 172.21.15.22:0/456607588 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698330030 0 0) 0x7fd8ec06e050 con 0x7fd8ec1454b0 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f0fc0640 1 -- 172.21.15.22:0/456607588 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d8003530 con 0x7fd8ec1454b0 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f27c3640 1 -- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 msgr2=0x7fd8ec072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f27c3640 1 -- 172.21.15.22:0/456607588 >> v1:172.21.15.183:6789/0 conn(0x7fd8ec1454b0 legacy=0x7fd8ec072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.539+0000 7fd8f27c3640 1 -- 172.21.15.22:0/456607588 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec0cedc0 con 0x7fd8ec145a00 2024-04-27T07:51:41.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec145a00 0x7fd8ec06edf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009d90 tx=0x7fd8e401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5aa7ec82260054 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f0fc0640 1 -- 172.21.15.22:0/456607588 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b2070 con 0x7fd8ec145a00 2024-04-27T07:51:41.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f0fc0640 1 -- 172.21.15.22:0/456607588 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e40a5830 con 0x7fd8ec145a00 2024-04-27T07:51:41.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f0fc0640 1 -- 172.21.15.22:0/456607588 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40a5be0 con 0x7fd8ec145a00 2024-04-27T07:51:41.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec145a00 msgr2=0x7fd8ec06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec145a00 0x7fd8ec06edf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009d90 tx=0x7fd8e401c970 comp rx=0 tx=0).stop 2024-04-27T07:51:41.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- 172.21.15.22:0/456607588 shutdown_connections 2024-04-27T07:51:41.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec145a00 0x7fd8ec06edf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- 172.21.15.22:0/456607588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- 172.21.15.22:0/456607588 >> 172.21.15.22:0/456607588 conn(0x7fd8ec067f10 msgr2=0x7fd8ec06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- 172.21.15.22:0/456607588 shutdown_connections 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- 172.21.15.22:0/456607588 wait complete. 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 Processor -- start 2024-04-27T07:51:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- start start 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 0x7fd8ec159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec15d690 con 0x7fd8ec145a00 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec15d510 con 0x7fd8ec1454b0 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f4226640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec15d810 con 0x7fd8ec072740 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f27c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f27c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45132/0 (socket says 172.21.15.22:45132) 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.543+0000 7fd8f27c3640 1 -- 172.21.15.22:0/2343728796 learned_addr learned my addr 172.21.15.22:0/2343728796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f17c1640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 0x7fd8ec159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f1fc2640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 -- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 msgr2=0x7fd8ec15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 -- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 msgr2=0x7fd8ec159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 0x7fd8ec159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 -- 172.21.15.22:0/2343728796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec13c820 con 0x7fd8ec145a00 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f17c1640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 0x7fd8ec159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f27c3640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40b0040 tx=0x7fd8e4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b2050 con 0x7fd8ec145a00 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e40be070 con 0x7fd8ec145a00 2024-04-27T07:51:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4003dd0 con 0x7fd8ec145a00 2024-04-27T07:51:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec13c540 con 0x7fd8ec145a00 2024-04-27T07:51:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec15a720 con 0x7fd8ec145a00 2024-04-27T07:51:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec068c30 con 0x7fd8ec145a00 2024-04-27T07:51:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd8e4004040 con 0x7fd8ec145a00 2024-04-27T07:51:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 0x7fd8d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd8e4121a60 con 0x7fd8ec145a00 2024-04-27T07:51:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f1fc2640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 0x7fd8d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.547+0000 7fd8f1fc2640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 0x7fd8d006a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8007fd0 tx=0x7fd8e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.551+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8e4016560 con 0x7fd8ec145a00 2024-04-27T07:51:41.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.743+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8ec06fa80 con 0x7fd8ec145a00 2024-04-27T07:51:41.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.743+0000 7fd8e27fc640 1 -- 172.21.15.22:0/2343728796 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fd8e40f1f90 con 0x7fd8ec145a00 2024-04-27T07:51:41.749 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:41.749 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 msgr2=0x7fd8d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 0x7fd8d006a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8007fd0 tx=0x7fd8e8009040 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 msgr2=0x7fd8ec1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40b0040 tx=0x7fd8e4019710 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 shutdown_connections 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd8d0068380 0x7fd8d006a840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ec145a00 0x7fd8ec1408e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec1454b0 0x7fd8ec159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 --2- 172.21.15.22:0/2343728796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8ec072740 0x7fd8ec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 >> 172.21.15.22:0/2343728796 conn(0x7fd8ec067f10 msgr2=0x7fd8ec072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 shutdown_connections 2024-04-27T07:51:41.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.747+0000 7fd8f4226640 1 -- 172.21.15.22:0/2343728796 wait complete. 2024-04-27T07:51:41.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.867+0000 7f7685328640 1 Processor -- start 2024-04-27T07:51:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.867+0000 7f7685328640 1 -- start start 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f7685328640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f768015ebe0 0x7f76800cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f7685328640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76800b3d30 con 0x7f76800b7930 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f7685328640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76800ba7a0 con 0x7f76800d4790 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f7685328640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f76800d4540 con 0x7f768015ebe0 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f76800b7930 0x7f76800d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59000/0 (socket says 172.21.15.22:59000) 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767f7fe640 1 -- 172.21.15.22:0/3900683366 learned_addr learned my addr 172.21.15.22:0/3900683366 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f768015ebe0 0x7f76800cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790570181 0 0) 0x7f76780014a0 con 0x7f76800b7930 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7668003530 con 0x7f76800b7930 2024-04-27T07:51:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3010806056 0 0) 0x7f7668003530 con 0x7f76800b7930 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f768015ebe0 msgr2=0x7f76800cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f768015ebe0 0x7f76800cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 >> v1:172.21.15.183:6789/0 conn(0x7f76800d4790 legacy=0x7f76800d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76800ba4c0 con 0x7f76800b7930 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7678001bd0 con 0x7f76800b7930 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 >> v1:172.21.15.22:6789/0 conn(0x7f76800b7930 legacy=0x7f76800d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76680039f0 0x7f7668003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7668004810 0x7f7668004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7668005190 0x7f766800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76800d4540 con 0x7f76680039f0 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f766800bf10 con 0x7f7668005190 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f766800c0e0 con 0x7f7668004810 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76680039f0 0x7f7668003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7668004810 0x7f7668004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767f7fe640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7668005190 0x7f766800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 -- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7668004810 msgr2=0x7f7668004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7668004810 0x7f7668004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 -- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7668005190 msgr2=0x7f766800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7668005190 0x7f766800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 -- 172.21.15.22:0/3900683366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7668003530 con 0x7f76680039f0 2024-04-27T07:51:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767f7fe640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7668005190 0x7f766800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7668004810 0x7f7668004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767effd640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76680039f0 0x7f7668003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7670001750 tx=0x7f767001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76419a66c0fb1852 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76700013a0 con 0x7f76680039f0 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76700052d0 con 0x7f76680039f0 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.871+0000 7f767dffb640 1 -- 172.21.15.22:0/3900683366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670003820 con 0x7f76680039f0 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3900683366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76680039f0 msgr2=0x7f7668003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 --2- 172.21.15.22:0/3900683366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76680039f0 0x7f7668003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7670001750 tx=0x7f767001c3b0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767f7fe640 1 -- 172.21.15.22:0/3900683366 reap_dead start 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3900683366 shutdown_connections 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3900683366 >> 172.21.15.22:0/3900683366 conn(0x7f7680067d10 msgr2=0x7f7680068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3900683366 shutdown_connections 2024-04-27T07:51:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3900683366 wait complete. 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 Processor -- start 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- start start 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76800ce030 0x7f76800b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 0x7f76800b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76800d3040 con 0x7f7680158180 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76800d2ec0 con 0x7f76800ce030 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f76800d31c0 con 0x7f76800b6ba0 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45164/0 (socket says 172.21.15.22:45164) 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 -- 172.21.15.22:0/3274516199 learned_addr learned my addr 172.21.15.22:0/3274516199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76800ce030 0x7f76800b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767f7fe640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 0x7f76800b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 -- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 msgr2=0x7f76800b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 0x7f76800b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 -- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76800ce030 msgr2=0x7f76800b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76800ce030 0x7f76800b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 -- 172.21.15.22:0/3274516199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7680158560 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767f7fe640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 0x7f76800b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f767effd640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f767001c1e0 tx=0x7f7668005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76700bc070 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670005480 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76800c91e0 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670007a60 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76800c97a0 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.875+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7680068c30 con 0x7f7680158180 2024-04-27T07:51:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.879+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f76700bf070 con 0x7f7680158180 2024-04-27T07:51:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.879+0000 7f76677fe640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 0x7f766c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.879+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f767011fcd0 con 0x7f7680158180 2024-04-27T07:51:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.879+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 0x7f766c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.879+0000 7f767e7fc640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 0x7f766c06a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7680161d70 tx=0x7f7674008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:41.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:41.883+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76700160f0 con 0x7f7680158180 2024-04-27T07:51:42.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.039+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7680075370 con 0x7f7680158180 2024-04-27T07:51:42.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.039+0000 7f76677fe640 1 -- 172.21.15.22:0/3274516199 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16774 (secure 0 0 0) 0x7f76700f0210 con 0x7f7680158180 2024-04-27T07:51:42.045 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:42.046 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:37.374737+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 msgr2=0x7f766c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 0x7f766c06a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7680161d70 tx=0x7f7674008040 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 msgr2=0x7f76800cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f767001c1e0 tx=0x7f7668005840 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 shutdown_connections 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f766c068380 0x7f766c06a840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76800b6ba0 0x7f76800b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76800ce030 0x7f76800b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 --2- 172.21.15.22:0/3274516199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7680158180 0x7f76800cdaf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 >> 172.21.15.22:0/3274516199 conn(0x7f7680067d10 msgr2=0x7f76800b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 shutdown_connections 2024-04-27T07:51:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.043+0000 7f7685328640 1 -- 172.21.15.22:0/3274516199 wait complete. 2024-04-27T07:51:42.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:50:46.370071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:50:47.097899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:42.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 Processor -- start 2024-04-27T07:51:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 -- start start 2024-04-27T07:51:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa024071f70 0x7fa024072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 0x7fa02406e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa024072890 con 0x7fa02406ec80 2024-04-27T07:51:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa0240ce0d0 con 0x7fa024071f70 2024-04-27T07:51:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa02b331640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa0240cd8c0 con 0x7fa0240cffe0 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa024071f70 0x7fa024072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa024071f70 0x7fa024072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45176/0 (socket says 172.21.15.22:45176) 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 -- 172.21.15.22:0/613365993 learned_addr learned my addr 172.21.15.22:0/613365993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0288cc640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 0x7fa02406e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 -- 172.21.15.22:0/613365993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 msgr2=0x7fa02406e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 0x7fa02406e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 -- 172.21.15.22:0/613365993 >> v1:172.21.15.183:6789/0 conn(0x7fa02406ec80 legacy=0x7fa02406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0290cd640 1 -- 172.21.15.22:0/613365993 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0240cdb10 con 0x7fa024071f70 2024-04-27T07:51:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.187+0000 7fa0288cc640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 0x7fa02406e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa0290cd640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa024072350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa014009e50 tx=0x7fa01401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44fb1a526f432f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa023fff640 1 -- 172.21.15.22:0/613365993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0140a5660 con 0x7fa024071f70 2024-04-27T07:51:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa023fff640 1 -- 172.21.15.22:0/613365993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0140a5800 con 0x7fa024071f70 2024-04-27T07:51:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa023fff640 1 -- 172.21.15.22:0/613365993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0140a5bd0 con 0x7fa024071f70 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 -- 172.21.15.22:0/613365993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 msgr2=0x7fa024072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa024072350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa014009e50 tx=0x7fa01401c9a0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 -- 172.21.15.22:0/613365993 shutdown_connections 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0240cffe0 0x7fa02406e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 --2- 172.21.15.22:0/613365993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa024072350 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 -- 172.21.15.22:0/613365993 >> 172.21.15.22:0/613365993 conn(0x7fa0240b21d0 msgr2=0x7fa0240b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 -- 172.21.15.22:0/613365993 shutdown_connections 2024-04-27T07:51:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.191+0000 7fa02b331640 1 -- 172.21.15.22:0/613365993 wait complete. 2024-04-27T07:51:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 Processor -- start 2024-04-27T07:51:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- start start 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa02406ec80 0x7fa02413bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0240cffe0 0x7fa02415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa024073130 con 0x7fa024071f70 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa024072fb0 con 0x7fa0240cffe0 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa0240cceb0 con 0x7fa02406ec80 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45190/0 (socket says 172.21.15.22:45190) 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 -- 172.21.15.22:0/2594564565 learned_addr learned my addr 172.21.15.22:0/2594564565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0298ce640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0240cffe0 0x7fa02415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0290cd640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa02406ec80 0x7fa02413bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 -- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa02406ec80 msgr2=0x7fa02413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa02406ec80 0x7fa02413bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 -- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0240cffe0 msgr2=0x7fa02415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0240cffe0 0x7fa02415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 -- 172.21.15.22:0/2594564565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0240a9b20 con 0x7fa024071f70 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa0288cc640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d800 tx=0x7fa01800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018018020 con 0x7fa024071f70 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0180096c0 con 0x7fa024071f70 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02415c5f0 con 0x7fa024071f70 2024-04-27T07:51:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018009ab0 con 0x7fa024071f70 2024-04-27T07:51:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa024137a70 con 0x7fa024071f70 2024-04-27T07:51:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02404ff10 con 0x7fa024071f70 2024-04-27T07:51:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa01801b020 con 0x7fa024071f70 2024-04-27T07:51:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.195+0000 7fa021ffb640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 0x7fa01006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.199+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa018086f70 con 0x7fa024071f70 2024-04-27T07:51:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.199+0000 7fa0290cd640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 0x7fa01006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.199+0000 7fa0290cd640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 0x7fa01006a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0140ae000 tx=0x7fa0140a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.203+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa018022020 con 0x7fa024071f70 2024-04-27T07:51:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.363+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa024072cb0 con 0x7fa024071f70 2024-04-27T07:51:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa021ffb640 1 -- 172.21.15.22:0/2594564565 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fa02415c5f0 con 0x7fa024071f70 2024-04-27T07:51:42.371 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 msgr2=0x7fa01006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 0x7fa01006a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0140ae000 tx=0x7fa0140a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 msgr2=0x7fa0240a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d800 tx=0x7fa01800dcc0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 shutdown_connections 2024-04-27T07:51:42.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa010068380 0x7fa01006a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0240cffe0 0x7fa02415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024071f70 0x7fa0240a9490 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 --2- 172.21.15.22:0/2594564565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa02406ec80 0x7fa02413bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 >> 172.21.15.22:0/2594564565 conn(0x7fa0240b21d0 msgr2=0x7fa02415d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 shutdown_connections 2024-04-27T07:51:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.367+0000 7fa02b331640 1 -- 172.21.15.22:0/2594564565 wait complete. 2024-04-27T07:51:42.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:51:42.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.507+0000 7fdefef90640 1 Processor -- start 2024-04-27T07:51:42.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefef90640 1 -- start start 2024-04-27T07:51:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefef90640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdef809b520 0x7fdef80b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefef90640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef80cea80 con 0x7fdef80b1f70 2024-04-27T07:51:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefef90640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef80b3700 con 0x7fdef809aa30 2024-04-27T07:51:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefef90640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdef80ced60 con 0x7fdef809b520 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefd52d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdef80b1f70 0x7fdef80b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59008/0 (socket says 172.21.15.22:59008) 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefd52d640 1 -- 172.21.15.22:0/3784501163 learned_addr learned my addr 172.21.15.22:0/3784501163 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef7fff640 1 --2- 172.21.15.22:0/3784501163 >> v2:172.21.15.183:3300/0 conn(0x7fdef809b520 0x7fdef80b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745723452 0 0) 0x7fdef80cea80 con 0x7fdef80b1f70 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdee0003530 con 0x7fdef80b1f70 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 782775792 0 0) 0x7fdee0003530 con 0x7fdef80b1f70 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 >> v2:172.21.15.183:3300/0 conn(0x7fdef809b520 msgr2=0x7fdef80b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 --2- 172.21.15.22:0/3784501163 >> v2:172.21.15.183:3300/0 conn(0x7fdef809b520 0x7fdef80b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 >> v1:172.21.15.183:6789/0 conn(0x7fdef809aa30 legacy=0x7fdef809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef80cdeb0 con 0x7fdef80b1f70 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fdef0001a00 con 0x7fdef80b1f70 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 >> v1:172.21.15.22:6789/0 conn(0x7fdef80b1f70 legacy=0x7fdef80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 0x7fdee0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee0004810 0x7fdee0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdee0005220 0x7fdee000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdef80ced60 con 0x7fdee00039c0 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdee000c170 con 0x7fdee0004810 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdee000c370 con 0x7fdee0005220 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 0x7fdee0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdef7fff640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee0004810 0x7fdee0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 -- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdee0005220 msgr2=0x7fdee000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdee0005220 0x7fdee000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 -- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee0004810 msgr2=0x7fdee0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee0004810 0x7fdee0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefcd2c640 1 -- 172.21.15.22:0/3784501163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef80cdeb0 con 0x7fdee00039c0 2024-04-27T07:51:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.511+0000 7fdefd52d640 1 -- 172.21.15.22:0/3784501163 reap_dead start 2024-04-27T07:51:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 0x7fdee0003de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdee8001750 tx=0x7fdee801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90abd1223db17b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80013a0 con 0x7fdee00039c0 2024-04-27T07:51:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee80052f0 con 0x7fdee00039c0 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdef77fe640 1 -- 172.21.15.22:0/3784501163 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8003840 con 0x7fdee00039c0 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 msgr2=0x7fdee0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 0x7fdee0003de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdee8001750 tx=0x7fdee801c3c0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- 172.21.15.22:0/3784501163 shutdown_connections 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 --2- 172.21.15.22:0/3784501163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdee00039c0 0x7fdee0003de0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- 172.21.15.22:0/3784501163 >> 172.21.15.22:0/3784501163 conn(0x7fdef80b4100 msgr2=0x7fdef80b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- 172.21.15.22:0/3784501163 shutdown_connections 2024-04-27T07:51:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- 172.21.15.22:0/3784501163 wait complete. 2024-04-27T07:51:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 Processor -- start 2024-04-27T07:51:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- start start 2024-04-27T07:51:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee00039c0 0x7fdef814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 0x7fdef80ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdef809b3a0 con 0x7fdef80cbc30 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdef809b220 con 0x7fdee00039c0 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefef90640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdef80ce3d0 con 0x7fdef80b7a40 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefd52d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefd52d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45208/0 (socket says 172.21.15.22:45208) 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefd52d640 1 -- 172.21.15.22:0/2753102236 learned_addr learned my addr 172.21.15.22:0/2753102236 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee00039c0 0x7fdef814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdef7fff640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 0x7fdef80ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.515+0000 7fdefd52d640 1 -- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 msgr2=0x7fdef80ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefd52d640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 0x7fdef80ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefd52d640 1 -- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee00039c0 msgr2=0x7fdef814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefd52d640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee00039c0 0x7fdef814ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefd52d640 1 -- 172.21.15.22:0/2753102236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef814f010 con 0x7fdef80cbc30 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef7fff640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 0x7fdef80ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefd52d640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b2350 tx=0x7fdef000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000cea0 con 0x7fdef80cbc30 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef000a610 con 0x7fdef80cbc30 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef80caa40 con 0x7fdef80cbc30 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000aa00 con 0x7fdef80cbc30 2024-04-27T07:51:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef80cb000 con 0x7fdef80cbc30 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef804ff10 con 0x7fdef80cbc30 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdef000aba0 con 0x7fdef80cbc30 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 0x7fdee406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdef0086210 con 0x7fdef80cbc30 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 0x7fdee406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.519+0000 7fdefcd2c640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 0x7fdee406a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdee0004790 tx=0x7fdee80011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.523+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdef0056750 con 0x7fdef80cbc30 2024-04-27T07:51:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.679+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdef8160700 con 0x7fdef80cbc30 2024-04-27T07:51:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.679+0000 7fdef57fa640 1 -- 172.21.15.22:0/2753102236 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fdef0056050 con 0x7fdef80cbc30 2024-04-27T07:51:42.685 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 msgr2=0x7fdee406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 0x7fdee406a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdee0004790 tx=0x7fdee80011a0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 msgr2=0x7fdef80cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdef80b2350 tx=0x7fdef000cc50 comp rx=0 tx=0).stop 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 shutdown_connections 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdee4068380 0x7fdee406a840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef80cbc30 0x7fdef80cc0e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdef80b7a40 0x7fdef80ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 --2- 172.21.15.22:0/2753102236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdee00039c0 0x7fdef814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 >> 172.21.15.22:0/2753102236 conn(0x7fdef80b4100 msgr2=0x7fdef8070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 shutdown_connections 2024-04-27T07:51:42.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.683+0000 7fdefef90640 1 -- 172.21.15.22:0/2753102236 wait complete. 2024-04-27T07:51:42.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:51:42.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 Processor -- start 2024-04-27T07:51:42.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 -- start start 2024-04-27T07:51:42.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1dc806f750 0x7f1dc8167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc80cdb10 con 0x7f1dc8071f70 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc80cddf0 con 0x7f1dc80cffe0 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcf011640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc80ce0d0 con 0x7f1dc806f750 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dccdad640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1dc8071f70 0x7f1dc8072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59014/0 (socket says 172.21.15.22:59014) 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dccdad640 1 -- 172.21.15.22:0/1725966921 learned_addr learned my addr 172.21.15.22:0/1725966921 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcd5ae640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1dc806f750 0x7f1dc8167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dc77fe640 1 -- 172.21.15.22:0/1725966921 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623764443 0 0) 0x7f1dc80cdb10 con 0x7f1dc8071f70 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dc77fe640 1 -- 172.21.15.22:0/1725966921 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1db0003530 con 0x7f1dc8071f70 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcd5ae640 1 -- 172.21.15.22:0/1725966921 >> v1:172.21.15.183:6789/0 conn(0x7f1dc80cffe0 legacy=0x7f1dc8164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcd5ae640 1 -- 172.21.15.22:0/1725966921 >> v1:172.21.15.22:6789/0 conn(0x7f1dc8071f70 legacy=0x7f1dc8072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.819+0000 7f1dcd5ae640 1 -- 172.21.15.22:0/1725966921 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc806e220 con 0x7f1dc806f750 2024-04-27T07:51:42.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcd5ae640 1 --2- 172.21.15.22:0/1725966921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc8167ac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009d90 tx=0x7f1dc001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a10d73426fe6f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dc77fe640 1 -- 172.21.15.22:0/1725966921 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00b2070 con 0x7f1dc806f750 2024-04-27T07:51:42.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dc77fe640 1 -- 172.21.15.22:0/1725966921 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc00a57a0 con 0x7f1dc806f750 2024-04-27T07:51:42.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dc77fe640 1 -- 172.21.15.22:0/1725966921 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00a5b70 con 0x7f1dc806f750 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- 172.21.15.22:0/1725966921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 msgr2=0x7f1dc8167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 --2- 172.21.15.22:0/1725966921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc8167ac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009d90 tx=0x7f1dc001c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- 172.21.15.22:0/1725966921 shutdown_connections 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 --2- 172.21.15.22:0/1725966921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc8167ac0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- 172.21.15.22:0/1725966921 >> 172.21.15.22:0/1725966921 conn(0x7f1dc80b21d0 msgr2=0x7f1dc80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- 172.21.15.22:0/1725966921 shutdown_connections 2024-04-27T07:51:42.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- 172.21.15.22:0/1725966921 wait complete. 2024-04-27T07:51:42.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 Processor -- start 2024-04-27T07:51:42.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- start start 2024-04-27T07:51:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dc8071f70 0x7f1dc815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dc80cffe0 0x7f1dc815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc806dfd0 con 0x7f1dc806f750 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80cd0c0 con 0x7f1dc8071f70 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dcf011640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc816a820 con 0x7f1dc80cffe0 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dccdad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.823+0000 7f1dccdad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45228/0 (socket says 172.21.15.22:45228) 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 -- 172.21.15.22:0/324135186 learned_addr learned my addr 172.21.15.22:0/324135186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dcd5ae640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dc80cffe0 0x7f1dc815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc7fff640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dc8071f70 0x7f1dc815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 -- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dc80cffe0 msgr2=0x7f1dc815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dc80cffe0 0x7f1dc815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 -- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dc8071f70 msgr2=0x7f1dc815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:42.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dc8071f70 0x7f1dc815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 -- 172.21.15.22:0/324135186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc8143090 con 0x7f1dc806f750 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dccdad640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c4d0 tx=0x7f1db800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dc806f750 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8005cf0 con 0x7f1dc806f750 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc815c220 con 0x7f1dc806f750 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007270 con 0x7f1dc806f750 2024-04-27T07:51:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc815c560 con 0x7f1dc806f750 2024-04-27T07:51:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc8072cb0 con 0x7f1dc806f750 2024-04-27T07:51:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1db801a070 con 0x7f1dc806f750 2024-04-27T07:51:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 0x7f1db406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1db8085d20 con 0x7f1dc806f750 2024-04-27T07:51:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc7fff640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 0x7f1db406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.827+0000 7f1dc7fff640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 0x7f1db406a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80cf940 tx=0x7f1dbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:42.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.831+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1db8055c10 con 0x7f1dc806f750 2024-04-27T07:51:42.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.991+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1dc804fef0 con 0x7f1dc806f750 2024-04-27T07:51:42.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.991+0000 7f1dc57fa640 1 -- 172.21.15.22:0/324135186 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f1db8059f10 con 0x7f1dc806f750 2024-04-27T07:51:42.998 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:51:43.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 msgr2=0x7f1db406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 0x7f1db406a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80cf940 tx=0x7f1dbc009040 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 msgr2=0x7f1dc815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c4d0 tx=0x7f1db800c990 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 shutdown_connections 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1db4068380 0x7f1db406a840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dc80cffe0 0x7f1dc815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dc8071f70 0x7f1dc815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 --2- 172.21.15.22:0/324135186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dc806f750 0x7f1dc815b750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 >> 172.21.15.22:0/324135186 conn(0x7f1dc80b21d0 msgr2=0x7f1dc80b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 shutdown_connections 2024-04-27T07:51:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:42.995+0000 7f1dcf011640 1 -- 172.21.15.22:0/324135186 wait complete. 2024-04-27T07:51:43.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:43.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.131+0000 7fd43f3fe640 1 Processor -- start 2024-04-27T07:51:43.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.131+0000 7fd43f3fe640 1 -- start start 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43f3fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd4380d4790 0x7fd4380d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43f3fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd43815ebe0 0x7fd4380cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43f3fe640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4380b3d30 con 0x7fd4380b7930 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43f3fe640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd4380d4540 con 0x7fd4380d4790 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43f3fe640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd4380ba7a0 con 0x7fd43815ebe0 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd4380d4790 0x7fd4380d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd4380d4790 0x7fd4380d4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45236/0 (socket says 172.21.15.22:45236) 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 -- 172.21.15.22:0/867295629 learned_addr learned my addr 172.21.15.22:0/867295629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:43.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43c999640 1 --2- 172.21.15.22:0/867295629 >> v2:172.21.15.183:3300/0 conn(0x7fd43815ebe0 0x7fd4380cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 -- 172.21.15.22:0/867295629 >> v2:172.21.15.183:3300/0 conn(0x7fd43815ebe0 msgr2=0x7fd4380cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:43.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 --2- 172.21.15.22:0/867295629 >> v2:172.21.15.183:3300/0 conn(0x7fd43815ebe0 0x7fd4380cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 -- 172.21.15.22:0/867295629 >> v1:172.21.15.22:6789/0 conn(0x7fd4380b7930 legacy=0x7fd4380b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 -- 172.21.15.22:0/867295629 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4380d50b0 con 0x7fd4380d4790 2024-04-27T07:51:43.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd42ffff640 1 -- 172.21.15.22:0/867295629 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368461759 0 0) 0x7fd4300014a0 con 0x7fd4380b7930 2024-04-27T07:51:43.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd43d19a640 1 --2- 172.21.15.22:0/867295629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4380d4790 0x7fd4380d4b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd4380d0b70 tx=0x7fd43401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35e3fb093d87fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd42ffff640 1 -- 172.21.15.22:0/867295629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340b0070 con 0x7fd4380d4790 2024-04-27T07:51:43.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd42ffff640 1 -- 172.21.15.22:0/867295629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4340ab600 con 0x7fd4380d4790 2024-04-27T07:51:43.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.135+0000 7fd42ffff640 1 -- 172.21.15.22:0/867295629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340ab9d0 con 0x7fd4380d4790 2024-04-27T07:51:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/867295629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4380d4790 msgr2=0x7fd4380d4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/867295629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4380d4790 0x7fd4380d4b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd4380d0b70 tx=0x7fd43401c490 comp rx=0 tx=0).stop 2024-04-27T07:51:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/867295629 shutdown_connections 2024-04-27T07:51:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/867295629 >> v2:172.21.15.183:3300/0 conn(0x7fd43815ebe0 0x7fd4380cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/867295629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4380d4790 0x7fd4380d4b70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/867295629 >> 172.21.15.22:0/867295629 conn(0x7fd438067d10 msgr2=0x7fd438068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/867295629 shutdown_connections 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/867295629 wait complete. 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 Processor -- start 2024-04-27T07:51:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- start start 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4380b7930 0x7fd438153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 0x7fd4381526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4380ba2a0 con 0x7fd43814cbf0 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4380ba120 con 0x7fd4380d4790 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd4380cfda0 con 0x7fd4380b7930 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45248/0 (socket says 172.21.15.22:45248) 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 -- 172.21.15.22:0/2049359242 learned_addr learned my addr 172.21.15.22:0/2049359242 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43c999640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 0x7fd4381526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d19a640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4380b7930 0x7fd438153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 -- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4380b7930 msgr2=0x7fd438153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4380b7930 0x7fd438153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 -- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 msgr2=0x7fd4381526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 0x7fd4381526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 -- 172.21.15.22:0/2049359242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438154640 con 0x7fd43814cbf0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43c999640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 0x7fd4381526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43d99b640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c820 tx=0x7fd43000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430017070 con 0x7fd43814cbf0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd430003dd0 con 0x7fd43814cbf0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4300052b0 con 0x7fd43814cbf0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.139+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd438154920 con 0x7fd43814cbf0 2024-04-27T07:51:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd438154ee0 con 0x7fd43814cbf0 2024-04-27T07:51:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd438068c30 con 0x7fd43814cbf0 2024-04-27T07:51:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd43001a070 con 0x7fd43814cbf0 2024-04-27T07:51:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd42dffb640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 0x7fd41c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd430085fe0 con 0x7fd43814cbf0 2024-04-27T07:51:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd43d19a640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 0x7fd41c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.143+0000 7fd43d19a640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 0x7fd41c06a870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4340ad000 tx=0x7fd4340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.147+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4380b3d30 con 0x7fd43814cbf0 2024-04-27T07:51:43.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.343+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4380cf5c0 con 0x7fd43814cbf0 2024-04-27T07:51:43.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd42dffb640 1 -- 172.21.15.22:0/2049359242 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fd430056590 con 0x7fd43814cbf0 2024-04-27T07:51:43.351 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:43.351 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 msgr2=0x7fd41c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 0x7fd41c06a870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4340ad000 tx=0x7fd4340a6000 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 msgr2=0x7fd438155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c820 tx=0x7fd43000cce0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 shutdown_connections 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd41c0683b0 0x7fd41c06a870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd43814cbf0 0x7fd438155860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4380d4790 0x7fd4381526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 --2- 172.21.15.22:0/2049359242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4380b7930 0x7fd438153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 >> 172.21.15.22:0/2049359242 conn(0x7fd438067d10 msgr2=0x7fd438152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 shutdown_connections 2024-04-27T07:51:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.347+0000 7fd43f3fe640 1 -- 172.21.15.22:0/2049359242 wait complete. 2024-04-27T07:51:43.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:43.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 Processor -- start 2024-04-27T07:51:43.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 -- start start 2024-04-27T07:51:43.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f934815ebe0 0x7f93480cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93480b7930 0x7f93480d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93480b3d30 con 0x7f93480d4790 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f93480d4540 con 0x7f93480b7930 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934ce01640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93480ba7a0 con 0x7f934815ebe0 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934659c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f93480d4790 0x7f93480d4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38142/0 (socket says 172.21.15.22:38142) 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934659c640 1 -- 172.21.15.22:0/811671531 learned_addr learned my addr 172.21.15.22:0/811671531 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9345d9b640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f934815ebe0 0x7f93480cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9346d9d640 1 --2- 172.21.15.22:0/811671531 >> v2:172.21.15.183:3300/0 conn(0x7f93480b7930 0x7f93480d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934559a640 1 -- 172.21.15.22:0/811671531 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862411304 0 0) 0x7f9338001120 con 0x7f93480d4790 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f934559a640 1 -- 172.21.15.22:0/811671531 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9330003530 con 0x7f93480d4790 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9346d9d640 1 -- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f934815ebe0 msgr2=0x7f93480cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9346d9d640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f934815ebe0 0x7f93480cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9346d9d640 1 -- 172.21.15.22:0/811671531 >> v1:172.21.15.183:6789/0 conn(0x7f93480d4790 legacy=0x7f93480d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.491+0000 7f9346d9d640 1 -- 172.21.15.22:0/811671531 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93480ba4c0 con 0x7f93480b7930 2024-04-27T07:51:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f9346d9d640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480b7930 0x7f93480d27b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9340009d90 tx=0x7f934001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd7c385296f598c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934559a640 1 -- 172.21.15.22:0/811671531 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400b2070 con 0x7f93480b7930 2024-04-27T07:51:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934559a640 1 -- 172.21.15.22:0/811671531 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93400a5970 con 0x7f93480b7930 2024-04-27T07:51:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934559a640 1 -- 172.21.15.22:0/811671531 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400a5d20 con 0x7f93480b7930 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480b7930 msgr2=0x7f93480d27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480b7930 0x7f93480d27b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9340009d90 tx=0x7f934001c8b0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- 172.21.15.22:0/811671531 shutdown_connections 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480b7930 0x7f93480d27b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 --2- 172.21.15.22:0/811671531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f934815ebe0 0x7f93480cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- 172.21.15.22:0/811671531 >> 172.21.15.22:0/811671531 conn(0x7f9348067d10 msgr2=0x7f9348068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- 172.21.15.22:0/811671531 shutdown_connections 2024-04-27T07:51:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- 172.21.15.22:0/811671531 wait complete. 2024-04-27T07:51:43.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 Processor -- start 2024-04-27T07:51:43.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 -- start start 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.495+0000 7f934ce01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480d4790 0x7f93480cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9348150490 0x7f93481544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93480cff20 con 0x7f93480b7930 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93480cfda0 con 0x7f93480d4790 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93480d3500 con 0x7f9348150490 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45256/0 (socket says 172.21.15.22:45256) 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 -- 172.21.15.22:0/877362282 learned_addr learned my addr 172.21.15.22:0/877362282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:43.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9346d9d640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9348150490 0x7f93481544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9345d9b640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480d4790 0x7f93480cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 -- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9348150490 msgr2=0x7f93481544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9348150490 0x7f93481544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 -- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480d4790 msgr2=0x7f93480cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480d4790 0x7f93480cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 -- 172.21.15.22:0/877362282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93480cc6c0 con 0x7f93480b7930 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934659c640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f933800c460 tx=0x7f933800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338017070 con 0x7f93480b7930 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9338005cf0 con 0x7f93480b7930 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93480cc9a0 con 0x7f93480b7930 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338007290 con 0x7f93480b7930 2024-04-27T07:51:43.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93480ccf60 con 0x7f93480b7930 2024-04-27T07:51:43.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9348068c30 con 0x7f93480b7930 2024-04-27T07:51:43.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f933801a070 con 0x7f93480b7930 2024-04-27T07:51:43.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 0x7f932806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9338085d80 con 0x7f93480b7930 2024-04-27T07:51:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9345d9b640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 0x7f932806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.499+0000 7f9345d9b640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 0x7f932806a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93480c83b0 tx=0x7f933c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.503+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f93480b3d30 con 0x7f93480b7930 2024-04-27T07:51:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.659+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f934815cea0 con 0x7f93480b7930 2024-04-27T07:51:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f9336ffd640 1 -- 172.21.15.22:0/877362282 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16774 (secure 0 0 0) 0x7f93380562b0 con 0x7f93480b7930 2024-04-27T07:51:43.668 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:43.668 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:37.374737+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 msgr2=0x7f932806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 0x7f932806a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93480c83b0 tx=0x7f933c008040 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 msgr2=0x7f934814aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f933800c460 tx=0x7f933800c920 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 shutdown_connections 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9328068380 0x7f932806a840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9348150490 0x7f93481544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93480d4790 0x7f93480cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 --2- 172.21.15.22:0/877362282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93480b7930 0x7f934814aea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 >> 172.21.15.22:0/877362282 conn(0x7f9348067d10 msgr2=0x7f934815c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 shutdown_connections 2024-04-27T07:51:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.663+0000 7f934ce01640 1 -- 172.21.15.22:0/877362282 wait complete. 2024-04-27T07:51:43.686 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:51:43.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.803+0000 7fcbe93a6640 1 Processor -- start 2024-04-27T07:51:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.803+0000 7fcbe93a6640 1 -- start start 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcbe40b4610 0x7fcbe406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe406c110 0x7fcbe40cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe40b6750 con 0x7fcbe4148230 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe40ceaf0 con 0x7fcbe40b4610 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe40b6190 con 0x7fcbe406c110 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcbe40b4610 0x7fcbe406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcbe40b4610 0x7fcbe406bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45258/0 (socket says 172.21.15.22:45258) 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 -- 172.21.15.22:0/1052050511 learned_addr learned my addr 172.21.15.22:0/1052050511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe37fe640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe406c110 0x7fcbe40cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 -- 172.21.15.22:0/1052050511 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe406c110 msgr2=0x7fcbe40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe406c110 0x7fcbe40cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 -- 172.21.15.22:0/1052050511 >> v1:172.21.15.22:6789/0 conn(0x7fcbe4148230 legacy=0x7fcbe40b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 -- 172.21.15.22:0/1052050511 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe40b6470 con 0x7fcbe40b4610 2024-04-27T07:51:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe1ffb640 1 -- 172.21.15.22:0/1052050511 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 220267471 0 0) 0x7fcbe40b6750 con 0x7fcbe4148230 2024-04-27T07:51:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe27fc640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe40b4610 0x7fcbe406bbd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aa8a16a5f773ab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe1ffb640 1 -- 172.21.15.22:0/1052050511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0b2070 con 0x7fcbe40b4610 2024-04-27T07:51:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe1ffb640 1 -- 172.21.15.22:0/1052050511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbcc0a9070 con 0x7fcbe40b4610 2024-04-27T07:51:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe1ffb640 1 -- 172.21.15.22:0/1052050511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0ad880 con 0x7fcbe40b4610 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- 172.21.15.22:0/1052050511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe40b4610 msgr2=0x7fcbe406bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe40b4610 0x7fcbe406bbd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc01c560 comp rx=0 tx=0).stop 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- 172.21.15.22:0/1052050511 shutdown_connections 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe406c110 0x7fcbe40cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/1052050511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe40b4610 0x7fcbe406bbd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- 172.21.15.22:0/1052050511 >> 172.21.15.22:0/1052050511 conn(0x7fcbe409aa30 msgr2=0x7fcbe40b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- 172.21.15.22:0/1052050511 shutdown_connections 2024-04-27T07:51:43.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.807+0000 7fcbe93a6640 1 -- 172.21.15.22:0/1052050511 wait complete. 2024-04-27T07:51:43.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 Processor -- start 2024-04-27T07:51:43.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- start start 2024-04-27T07:51:43.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbe40b4610 0x7fcbe4141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe4148230 0x7fcbe41415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe40d00e0 con 0x7fcbe406c110 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe40cff60 con 0x7fcbe40b4610 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe40d0260 con 0x7fcbe4148230 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45268/0 (socket says 172.21.15.22:45268) 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 -- 172.21.15.22:0/2282320677 learned_addr learned my addr 172.21.15.22:0/2282320677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 -- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe4148230 msgr2=0x7fcbe41415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe4148230 0x7fcbe41415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 -- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbe40b4610 msgr2=0x7fcbe4141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbe40b4610 0x7fcbe4141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:43.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 -- 172.21.15.22:0/2282320677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe415efd0 con 0x7fcbe406c110 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe2ffd640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c4d0 tx=0x7fcbd400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4017070 con 0x7fcbe406c110 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd4003600 con 0x7fcbe406c110 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe4141de0 con 0x7fcbe406c110 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40054a0 con 0x7fcbe406c110 2024-04-27T07:51:43.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe413a4c0 con 0x7fcbe406c110 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe4139fd0 con 0x7fcbe406c110 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.811+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcbd401a070 con 0x7fcbe406c110 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.815+0000 7fcbdb7fe640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 0x7fcbc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.815+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcbd4085e70 con 0x7fcbe406c110 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.815+0000 7fcbe27fc640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 0x7fcbc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.815+0000 7fcbe27fc640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 0x7fcbc406a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0af000 tx=0x7fcbcc004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:43.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:43.819+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcbd4056470 con 0x7fcbe406c110 2024-04-27T07:51:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.010+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbe4069340 con 0x7fcbe406c110 2024-04-27T07:51:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbdb7fe640 1 -- 172.21.15.22:0/2282320677 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fcbd4055d70 con 0x7fcbe406c110 2024-04-27T07:51:44.020 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:44.020 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 msgr2=0x7fcbc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 0x7fcbc406a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0af000 tx=0x7fcbcc004740 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 msgr2=0x7fcbe4143730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c4d0 tx=0x7fcbd400c990 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 shutdown_connections 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcbc4068380 0x7fcbc406a840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbe4148230 0x7fcbe41415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbe40b4610 0x7fcbe4141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 --2- 172.21.15.22:0/2282320677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbe406c110 0x7fcbe4143730 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 >> 172.21.15.22:0/2282320677 conn(0x7fcbe409aa30 msgr2=0x7fcbe40b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 shutdown_connections 2024-04-27T07:51:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.014+0000 7fcbe93a6640 1 -- 172.21.15.22:0/2282320677 wait complete. 2024-04-27T07:51:44.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:44.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 Processor -- start 2024-04-27T07:51:44.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 -- start start 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcf7006fc70 0x7fcf7006e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 0x7fcf700b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7006e630 con 0x7fcf70162da0 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcf70163180 con 0x7fcf7006fc70 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf7755b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7006f1b0 con 0x7fcf700ce260 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf74af6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcf7006fc70 0x7fcf7006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf74af6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcf7006fc70 0x7fcf7006e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45272/0 (socket says 172.21.15.22:45272) 2024-04-27T07:51:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.150+0000 7fcf74af6640 1 -- 172.21.15.22:0/2677393086 learned_addr learned my addr 172.21.15.22:0/2677393086 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf75af8640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 0x7fcf700b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf74af6640 1 -- 172.21.15.22:0/2677393086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 msgr2=0x7fcf700b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf74af6640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 0x7fcf700b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf74af6640 1 -- 172.21.15.22:0/2677393086 >> v1:172.21.15.183:6789/0 conn(0x7fcf70162da0 legacy=0x7fcf7006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf74af6640 1 -- 172.21.15.22:0/2677393086 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7006e910 con 0x7fcf7006fc70 2024-04-27T07:51:44.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf75af8640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 0x7fcf700b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf74af6640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf7006e050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009870 tx=0x7fcf6001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c579e7d276e11566 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf67fff640 1 -- 172.21.15.22:0/2677393086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600b2070 con 0x7fcf7006fc70 2024-04-27T07:51:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf67fff640 1 -- 172.21.15.22:0/2677393086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf600a9070 con 0x7fcf7006fc70 2024-04-27T07:51:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf67fff640 1 -- 172.21.15.22:0/2677393086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600ad880 con 0x7fcf7006fc70 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf7755b640 1 -- 172.21.15.22:0/2677393086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 msgr2=0x7fcf7006e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.154+0000 7fcf7755b640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf7006e050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009870 tx=0x7fcf6001c560 comp rx=0 tx=0).stop 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/2677393086 shutdown_connections 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf700ce260 0x7fcf700b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 --2- 172.21.15.22:0/2677393086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf7006e050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/2677393086 >> 172.21.15.22:0/2677393086 conn(0x7fcf70067f10 msgr2=0x7fcf7006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/2677393086 shutdown_connections 2024-04-27T07:51:44.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/2677393086 wait complete. 2024-04-27T07:51:44.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 Processor -- start 2024-04-27T07:51:44.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- start start 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf700ce260 0x7fcf701532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf70162da0 0x7fcf7015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700b5700 con 0x7fcf7006fc70 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700b5580 con 0x7fcf700ce260 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700b5880 con 0x7fcf70162da0 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45282/0 (socket says 172.21.15.22:45282) 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 -- 172.21.15.22:0/936133393 learned_addr learned my addr 172.21.15.22:0/936133393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf74af6640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf700ce260 0x7fcf701532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf75af8640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf70162da0 0x7fcf7015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 -- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf70162da0 msgr2=0x7fcf7015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf70162da0 0x7fcf7015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 -- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf700ce260 msgr2=0x7fcf701532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf700ce260 0x7fcf701532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 -- 172.21.15.22:0/936133393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701537e0 con 0x7fcf7006fc70 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf752f7640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c420 tx=0x7fcf6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680052d0 con 0x7fcf7006fc70 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68005470 con 0x7fcf7006fc70 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf70157ef0 con 0x7fcf7006fc70 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68007260 con 0x7fcf7006fc70 2024-04-27T07:51:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701584b0 con 0x7fcf7006fc70 2024-04-27T07:51:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.158+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70068c30 con 0x7fcf7006fc70 2024-04-27T07:51:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.162+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcf68007400 con 0x7fcf7006fc70 2024-04-27T07:51:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.162+0000 7fcf65ffb640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 0x7fcf5406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.162+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcf68085ed0 con 0x7fcf7006fc70 2024-04-27T07:51:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.162+0000 7fcf74af6640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 0x7fcf5406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.162+0000 7fcf74af6640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 0x7fcf5406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf600af000 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.166+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf680564a0 con 0x7fcf7006fc70 2024-04-27T07:51:44.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.354+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf700cea80 con 0x7fcf7006fc70 2024-04-27T07:51:44.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.354+0000 7fcf65ffb640 1 -- 172.21.15.22:0/936133393 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fcf68055da0 con 0x7fcf7006fc70 2024-04-27T07:51:44.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:44.363 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/3811508140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3811508140},{"type":"v1","addr":"172.21.15.22:6833","nonce":3811508140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3534337368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3534337368},{"type":"v1","addr":"172.21.15.22:6835","nonce":3534337368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/2269042699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2269042699},{"type":"v1","addr":"172.21.15.183:6836","nonce":2269042699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:50:48.129218+0000","modified":"2024-04-27T07:50:51.161261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/1295572944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1295572944},{"type":"v1","addr":"172.21.15.183:6837","nonce":1295572944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-27T07:51:44.363 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-27T07:51:44.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 msgr2=0x7fcf5406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 0x7fcf5406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf600af000 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 msgr2=0x7fcf70158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c420 tx=0x7fcf6800c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 shutdown_connections 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcf54068380 0x7fcf5406a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf70162da0 0x7fcf7015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf700ce260 0x7fcf701532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 --2- 172.21.15.22:0/936133393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf7006fc70 0x7fcf70158f90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 >> 172.21.15.22:0/936133393 conn(0x7fcf70067f10 msgr2=0x7fcf700cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 shutdown_connections 2024-04-27T07:51:44.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.358+0000 7fcf7755b640 1 -- 172.21.15.22:0/936133393 wait complete. 2024-04-27T07:51:44.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:44.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 Processor -- start 2024-04-27T07:51:44.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 -- start start 2024-04-27T07:51:44.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f200d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f2015ebe0 0x7f7f200cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f200b3d30 con 0x7f7f200b7930 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7f200d4540 con 0x7f7f2015ebe0 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f25832640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f200ba7a0 con 0x7f7f200d4790 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7f200b7930 0x7f7f200b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59034/0 (socket says 172.21.15.22:59034) 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 -- 172.21.15.22:0/4006132882 learned_addr learned my addr 172.21.15.22:0/4006132882 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1e7fc640 1 --2- 172.21.15.22:0/4006132882 >> v2:172.21.15.183:3300/0 conn(0x7f7f2015ebe0 0x7f7f200cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1effd640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f200d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243648712 0 0) 0x7f7f10001120 con 0x7f7f200b7930 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0c003530 con 0x7f7f200b7930 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1196450615 0 0) 0x7f7f0c003530 con 0x7f7f200b7930 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 msgr2=0x7f7f200d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f200d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 >> v2:172.21.15.183:3300/0 conn(0x7f7f2015ebe0 msgr2=0x7f7f200cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 --2- 172.21.15.22:0/4006132882 >> v2:172.21.15.183:3300/0 conn(0x7f7f2015ebe0 0x7f7f200cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f200d50b0 con 0x7f7f200b7930 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7f10001870 con 0x7f7f200b7930 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 >> v1:172.21.15.22:6789/0 conn(0x7f7f200b7930 legacy=0x7f7f200b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f0c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f0c003f90 0x7f7f0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f0c004d50 0x7f7f0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200d4540 con 0x7f7f0c004d50 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200ba7a0 con 0x7f7f0c003f90 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c00ba80 con 0x7f7f200d4790 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f0c004d50 0x7f7f0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1e7fc640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f0c003f90 0x7f7f0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1effd640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f0c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 -- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 msgr2=0x7f7f0c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f0c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 -- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f0c003f90 msgr2=0x7f7f0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f0c003f90 0x7f7f0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1f7fe640 1 -- 172.21.15.22:0/4006132882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c003530 con 0x7f7f0c004d50 2024-04-27T07:51:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.498+0000 7f7f1effd640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f200d4790 0x7f7f0c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:44.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f1f7fe640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f0c004d50 0x7f7f0c0051d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c490 tx=0x7f7f1001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0741ee0ed8370d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000c0e0 con 0x7f7f0c004d50 2024-04-27T07:51:44.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f10005b00 con 0x7f7f0c004d50 2024-04-27T07:51:44.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f1dffb640 1 -- 172.21.15.22:0/4006132882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100070f0 con 0x7f7f0c004d50 2024-04-27T07:51:44.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 -- 172.21.15.22:0/4006132882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f0c004d50 msgr2=0x7f7f0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 --2- 172.21.15.22:0/4006132882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f0c004d50 0x7f7f0c0051d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c490 tx=0x7f7f1001f330 comp rx=0 tx=0).stop 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f1f7fe640 1 -- 172.21.15.22:0/4006132882 reap_dead start 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 -- 172.21.15.22:0/4006132882 shutdown_connections 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 -- 172.21.15.22:0/4006132882 >> 172.21.15.22:0/4006132882 conn(0x7f7f20067d10 msgr2=0x7f7f20068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 -- 172.21.15.22:0/4006132882 shutdown_connections 2024-04-27T07:51:44.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.502+0000 7f7f25832640 1 -- 172.21.15.22:0/4006132882 wait complete. 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 Processor -- start 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- start start 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f201f43c0 0x7f7f20160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f201f4ce0 0x7f7f200b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200d4260 con 0x7f7f20160f70 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200d3f80 con 0x7f7f201f43c0 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f200ba120 con 0x7f7f201f4ce0 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45294/0 (socket says 172.21.15.22:45294) 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 -- 172.21.15.22:0/2179447769 learned_addr learned my addr 172.21.15.22:0/2179447769 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1effd640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f201f43c0 0x7f7f20160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1f7fe640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f201f4ce0 0x7f7f200b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 -- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f201f4ce0 msgr2=0x7f7f200b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f201f4ce0 0x7f7f200b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 -- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f201f43c0 msgr2=0x7f7f20160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f201f43c0 0x7f7f20160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 -- 172.21.15.22:0/2179447769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f201616f0 con 0x7f7f20160f70 2024-04-27T07:51:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f1e7fc640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009f90 tx=0x7f7f0800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800fea0 con 0x7f7f20160f70 2024-04-27T07:51:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f08015630 con 0x7f7f20160f70 2024-04-27T07:51:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08015a20 con 0x7f7f20160f70 2024-04-27T07:51:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f200b4f70 con 0x7f7f20160f70 2024-04-27T07:51:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201613f0 con 0x7f7f20160f70 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f20068c30 con 0x7f7f20160f70 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7f08015bc0 con 0x7f7f20160f70 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 0x7f7f0006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.506+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7f08087b20 con 0x7f7f20160f70 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.510+0000 7f7f1effd640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 0x7f7f0006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.510+0000 7f7f1effd640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 0x7f7f0006a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f20161ce0 tx=0x7f7f18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.514+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f08019030 con 0x7f7f20160f70 2024-04-27T07:51:44.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.674+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f200b9bc0 con 0x7f7f20160f70 2024-04-27T07:51:44.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.674+0000 7f7f17fff640 1 -- 172.21.15.22:0/2179447769 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16774 (secure 0 0 0) 0x7f7f08010040 con 0x7f7f20160f70 2024-04-27T07:51:44.683 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:44.683 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:37.374737+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:50:46.370071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:50:47.097899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 msgr2=0x7f7f0006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 0x7f7f0006a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f20161ce0 tx=0x7f7f18008040 comp rx=0 tx=0).stop 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 msgr2=0x7f7f201f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009f90 tx=0x7f7f0800f580 comp rx=0 tx=0).stop 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 shutdown_connections 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7f000683b0 0x7f7f0006a870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f201f4ce0 0x7f7f200b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f20160f70 0x7f7f201f47a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 --2- 172.21.15.22:0/2179447769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f201f43c0 0x7f7f20160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 >> 172.21.15.22:0/2179447769 conn(0x7f7f20067d10 msgr2=0x7f7f2015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 shutdown_connections 2024-04-27T07:51:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.678+0000 7f7f25832640 1 -- 172.21.15.22:0/2179447769 wait complete. 2024-04-27T07:51:44.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-27T07:51:44.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 Processor -- start 2024-04-27T07:51:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 -- start start 2024-04-27T07:51:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8ce4144dd0 0x7f8ce406f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 0x7f8ce40cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8ce40cfc40 0x7f8ce415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.806+0000 7f8cea0c3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce406fb00 con 0x7f8ce4144dd0 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce40720d0 con 0x7f8ce40cfc40 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce4072370 con 0x7f8ce406f6a0 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8ce4144dd0 0x7f8ce406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8ce4144dd0 0x7f8ce406f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45310/0 (socket says 172.21.15.22:45310) 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 -- 172.21.15.22:0/2300354798 learned_addr learned my addr 172.21.15.22:0/2300354798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce3fff640 1 --2- 172.21.15.22:0/2300354798 >> v2:172.21.15.183:3300/0 conn(0x7f8ce40cfc40 0x7f8ce415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cdbfff640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 0x7f8ce40cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 -- 172.21.15.22:0/2300354798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 msgr2=0x7f8ce40cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 0x7f8ce40cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 -- 172.21.15.22:0/2300354798 >> v2:172.21.15.183:3300/0 conn(0x7f8ce40cfc40 msgr2=0x7f8ce415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2300354798 >> v2:172.21.15.183:3300/0 conn(0x7f8ce40cfc40 0x7f8ce415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 -- 172.21.15.22:0/2300354798 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce4072650 con 0x7f8ce4144dd0 2024-04-27T07:51:44.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cdbfff640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 0x7f8ce40cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce4144dd0 0x7f8ce406f160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ce406cc60 tx=0x7f8cd401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe47db9e4665e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/2300354798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40b1070 con 0x7f8ce4144dd0 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/2300354798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd40ac630 con 0x7f8ce4144dd0 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/2300354798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40a8050 con 0x7f8ce4144dd0 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2300354798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce4144dd0 msgr2=0x7f8ce406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce4144dd0 0x7f8ce406f160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ce406cc60 tx=0x7f8cd401c520 comp rx=0 tx=0).stop 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2300354798 shutdown_connections 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2300354798 >> v2:172.21.15.183:3300/0 conn(0x7f8ce40cfc40 0x7f8ce415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce406f6a0 0x7f8ce40cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2300354798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce4144dd0 0x7f8ce406f160 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2300354798 >> 172.21.15.22:0/2300354798 conn(0x7f8ce4067f10 msgr2=0x7f8ce406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:44.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2300354798 shutdown_connections 2024-04-27T07:51:44.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.810+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2300354798 wait complete. 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 Processor -- start 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- start start 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 0x7f8ce4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce4144dd0 0x7f8ce413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce40d01d0 con 0x7f8ce40cfc40 2024-04-27T07:51:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce40d0050 con 0x7f8ce406f6a0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce40b3dd0 con 0x7f8ce4144dd0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45320/0 (socket says 172.21.15.22:45320) 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 -- 172.21.15.22:0/2086496717 learned_addr learned my addr 172.21.15.22:0/2086496717 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce3fff640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce4144dd0 0x7f8ce413ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 0x7f8ce4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 -- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce4144dd0 msgr2=0x7f8ce413ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce4144dd0 0x7f8ce413ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 -- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 msgr2=0x7f8ce4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 0x7f8ce4158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 -- 172.21.15.22:0/2086496717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce413b300 con 0x7f8ce40cfc40 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 0x7f8ce4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cdbfff640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c7f0 tx=0x7f8ccc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc007a30 con 0x7f8ce40cfc40 2024-04-27T07:51:44.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc007bd0 con 0x7f8ce40cfc40 2024-04-27T07:51:44.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0113f0 con 0x7f8ce40cfc40 2024-04-27T07:51:44.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce413efd0 con 0x7f8ce40cfc40 2024-04-27T07:51:44.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce413f550 con 0x7f8ce40cfc40 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce40a9490 con 0x7f8ce40cfc40 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ccc011590 con 0x7f8ce40cfc40 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 0x7f8cc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.814+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8ccc086c70 con 0x7f8ce40cfc40 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.818+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 0x7f8cc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:44.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.818+0000 7f8ce37fe640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 0x7f8cc406a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40ae000 tx=0x7f8cd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:44.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:44.822+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ccc015020 con 0x7f8ce40cfc40 2024-04-27T07:51:45.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.018+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8ce406e860 con 0x7f8ce40cfc40 2024-04-27T07:51:45.024 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-04-27T07:51:45.018+0000 7fadb8a64640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T07:51:45.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.018+0000 7f8ce0ff9640 1 -- 172.21.15.22:0/2086496717 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f8ccc0571b0 con 0x7f8ce40cfc40 2024-04-27T07:51:45.026 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-27T07:51:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 msgr2=0x7f8cc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 0x7f8cc406a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40ae000 tx=0x7f8cd4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 msgr2=0x7f8ce413adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c7f0 tx=0x7f8ccc00ccb0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 shutdown_connections 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8cc4068380 0x7f8cc406a840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ce4144dd0 0x7f8ce413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ce40cfc40 0x7f8ce413adc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 --2- 172.21.15.22:0/2086496717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ce406f6a0 0x7f8ce4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 >> 172.21.15.22:0/2086496717 conn(0x7f8ce4067f10 msgr2=0x7f8ce406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 shutdown_connections 2024-04-27T07:51:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.022+0000 7f8cea0c3640 1 -- 172.21.15.22:0/2086496717 wait complete. 2024-04-27T07:51:45.041 INFO:tasks.ceph.mds.b.smithi183.stderr: -3500> 2024-04-27T07:50:52.121+0000 7f1ec98f5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:51:45.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-27T07:51:45.094 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2024-04-27T07:51:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 Processor -- start 2024-04-27T07:51:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 -- start start 2024-04-27T07:51:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f76ec161f30 0x7f76ec0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec0afd00 0x7f76ec06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ec06e070 con 0x7f76ec071f70 2024-04-27T07:51:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f76ec072420 con 0x7f76ec161f30 2024-04-27T07:51:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76f1da2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f76ec0726c0 con 0x7f76ec0afd00 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76ebfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec0afd00 0x7f76ec06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eb7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f76ec161f30 0x7f76ec0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eaffd640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f76ec071f70 0x7f76ec0af5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:38162/0 (socket says 172.21.15.22:38162) 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eaffd640 1 -- 172.21.15.22:0/1180396198 learned_addr learned my addr 172.21.15.22:0/1180396198 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eb7fe640 1 -- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec0afd00 msgr2=0x7f76ec06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec0afd00 0x7f76ec06e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eb7fe640 1 -- 172.21.15.22:0/1180396198 >> v1:172.21.15.183:6789/0 conn(0x7f76ec071f70 legacy=0x7f76ec0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.162+0000 7f76eb7fe640 1 -- 172.21.15.22:0/1180396198 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec0729a0 con 0x7f76ec161f30 2024-04-27T07:51:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec161f30 0x7f76ec0b34d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a310791010b8e314 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76ea7fc640 1 -- 172.21.15.22:0/1180396198 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0b0070 con 0x7f76ec161f30 2024-04-27T07:51:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76ea7fc640 1 -- 172.21.15.22:0/1180396198 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc0ab600 con 0x7f76ec161f30 2024-04-27T07:51:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76ea7fc640 1 -- 172.21.15.22:0/1180396198 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0ab9d0 con 0x7f76ec161f30 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec161f30 msgr2=0x7f76ec0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 --2- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec161f30 0x7f76ec0b34d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c410 comp rx=0 tx=0).stop 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- 172.21.15.22:0/1180396198 shutdown_connections 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 --2- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec0afd00 0x7f76ec06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 --2- 172.21.15.22:0/1180396198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec161f30 0x7f76ec0b34d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- 172.21.15.22:0/1180396198 >> 172.21.15.22:0/1180396198 conn(0x7f76ec0b21d0 msgr2=0x7f76ec0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- 172.21.15.22:0/1180396198 shutdown_connections 2024-04-27T07:51:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- 172.21.15.22:0/1180396198 wait complete. 2024-04-27T07:51:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 Processor -- start 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.166+0000 7f76f1da2640 1 -- start start 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 0x7f76ec147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec161f30 0x7f76ec0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec06f6d0 con 0x7f76ec071f70 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec06f550 con 0x7f76ec0afd00 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f76ec06f850 con 0x7f76ec161f30 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45332/0 (socket says 172.21.15.22:45332) 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2664525279 learned_addr learned my addr 172.21.15.22:0/2664525279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 0x7f76ec147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76ebfff640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec161f30 0x7f76ec0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec161f30 msgr2=0x7f76ec0d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec161f30 0x7f76ec0d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 msgr2=0x7f76ec147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 0x7f76ec147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2664525279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec147940 con 0x7f76ec071f70 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 0x7f76ec147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0b0040 con 0x7f76ec071f70 2024-04-27T07:51:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc0ba070 con 0x7f76ec071f70 2024-04-27T07:51:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec147c20 con 0x7f76ec071f70 2024-04-27T07:51:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc006e20 con 0x7f76ec071f70 2024-04-27T07:51:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec1481e0 con 0x7f76ec071f70 2024-04-27T07:51:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ec04ff10 con 0x7f76ec071f70 2024-04-27T07:51:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f76dc0b4630 con 0x7f76ec071f70 2024-04-27T07:51:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 0x7f76d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f76dc120750 con 0x7f76ec071f70 2024-04-27T07:51:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.170+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 0x7f76d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.174+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 0x7f76d806a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76ec153a80 tx=0x7f76e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:45.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.174+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76dc016090 con 0x7f76ec071f70 2024-04-27T07:51:45.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:45.366+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f76ec0b0420 con 0x7f76ec071f70 2024-04-27T07:51:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.022+0000 7f76d3fff640 1 -- 172.21.15.22:0/2664525279 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f76dc0f0ba0 con 0x7f76ec071f70 2024-04-27T07:51:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 msgr2=0x7f76d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 0x7f76d806a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76ec153a80 tx=0x7f76e0009040 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 msgr2=0x7f76ec15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc0191b0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 shutdown_connections 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f76d80683b0 0x7f76d806a870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76ec161f30 0x7f76ec0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76ec0afd00 0x7f76ec147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 --2- 172.21.15.22:0/2664525279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec071f70 0x7f76ec15a510 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 >> 172.21.15.22:0/2664525279 conn(0x7f76ec0b21d0 msgr2=0x7f76ec159280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 shutdown_connections 2024-04-27T07:51:46.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.026+0000 7f76f1da2640 1 -- 172.21.15.22:0/2664525279 wait complete. 2024-04-27T07:51:46.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-27T07:51:46.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.182+0000 7f08ba727640 1 Processor -- start 2024-04-27T07:51:46.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.182+0000 7f08ba727640 1 -- start start 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08ba727640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f08b409aa30 0x7f08b40b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08ba727640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b40b1f70 0x7f08b40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08ba727640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b40cea80 con 0x7f08b409ae10 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08ba727640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f08b40ced60 con 0x7f08b409aa30 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08ba727640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f08b40b3700 con 0x7f08b40b1f70 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f08b409aa30 0x7f08b40b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b37fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f08b409ae10 0x7f08b409b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59038/0 (socket says 172.21.15.22:59038) 2024-04-27T07:51:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b37fe640 1 -- 172.21.15.22:0/1798253211 learned_addr learned my addr 172.21.15.22:0/1798253211 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b8cc4640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b40b1f70 0x7f08b40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b3fff640 1 -- 172.21.15.22:0/1798253211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b40b1f70 msgr2=0x7f08b40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b3fff640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b40b1f70 0x7f08b40b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b3fff640 1 -- 172.21.15.22:0/1798253211 >> v1:172.21.15.22:6789/0 conn(0x7f08b409ae10 legacy=0x7f08b409b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b3fff640 1 -- 172.21.15.22:0/1798253211 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b40cdeb0 con 0x7f08b409aa30 2024-04-27T07:51:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.186+0000 7f08b2ffd640 1 -- 172.21.15.22:0/1798253211 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607690680 0 0) 0x7f08b40cea80 con 0x7f08b409ae10 2024-04-27T07:51:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b3fff640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b409aa30 0x7f08b40b39e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f08a40097c0 tx=0x7f08a401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12631c1e880a10d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b2ffd640 1 -- 172.21.15.22:0/1798253211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b1070 con 0x7f08b409aa30 2024-04-27T07:51:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b2ffd640 1 -- 172.21.15.22:0/1798253211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a40ac600 con 0x7f08b409aa30 2024-04-27T07:51:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b2ffd640 1 -- 172.21.15.22:0/1798253211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40ac9d0 con 0x7f08b409aa30 2024-04-27T07:51:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- 172.21.15.22:0/1798253211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b409aa30 msgr2=0x7f08b40b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b409aa30 0x7f08b40b39e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f08a40097c0 tx=0x7f08a401c380 comp rx=0 tx=0).stop 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- 172.21.15.22:0/1798253211 shutdown_connections 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b40b1f70 0x7f08b40b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- 172.21.15.22:0/1798253211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b409aa30 0x7f08b40b39e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- 172.21.15.22:0/1798253211 >> 172.21.15.22:0/1798253211 conn(0x7f08b40b4100 msgr2=0x7f08b40b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- 172.21.15.22:0/1798253211 shutdown_connections 2024-04-27T07:51:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- 172.21.15.22:0/1798253211 wait complete. 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 Processor -- start 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- start start 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b409aa30 0x7f08b4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 0x7f08b4158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08b40ce550 con 0x7f08b40b1f70 2024-04-27T07:51:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08b40ce3d0 con 0x7f08b409aa30 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08ba727640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f08b4169df0 con 0x7f08b409ae10 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45346/0 (socket says 172.21.15.22:45346) 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 -- 172.21.15.22:0/1185042119 learned_addr learned my addr 172.21.15.22:0/1185042119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b3fff640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b409aa30 0x7f08b4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b37fe640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 0x7f08b4158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 -- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 msgr2=0x7f08b4158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 0x7f08b4158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 -- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b409aa30 msgr2=0x7f08b4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b409aa30 0x7f08b4157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 -- 172.21.15.22:0/1185042119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b4159200 con 0x7f08b40b1f70 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b37fe640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 0x7f08b4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.190+0000 7f08b8cc4640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f08ac007e40 tx=0x7f08ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac014620 con 0x7f08b40b1f70 2024-04-27T07:51:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ac019070 con 0x7f08b40b1f70 2024-04-27T07:51:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b41594e0 con 0x7f08b40b1f70 2024-04-27T07:51:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac014a20 con 0x7f08b40b1f70 2024-04-27T07:51:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b4159aa0 con 0x7f08b40b1f70 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b404ff10 con 0x7f08b40b1f70 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f08ac014bc0 con 0x7f08b40b1f70 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 0x7f08a006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f08ac0872a0 con 0x7f08b40b1f70 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b3fff640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 0x7f08a006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.194+0000 7f08b3fff640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 0x7f08a006a870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ae000 tx=0x7f08a40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.198+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08ac057820 con 0x7f08b40b1f70 2024-04-27T07:51:46.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:46.366+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08b40cd4a0 con 0x7f08b40b1f70 2024-04-27T07:51:47.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.030+0000 7f08b0ff9640 1 -- 172.21.15.22:0/1185042119 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f08ac057120 con 0x7f08b40b1f70 2024-04-27T07:51:47.037 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-04-27T07:51:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.034+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 msgr2=0x7f08a006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.034+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 0x7f08a006a870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ae000 tx=0x7f08a40a7000 comp rx=0 tx=0).stop 2024-04-27T07:51:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.034+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 msgr2=0x7f08b40701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.034+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f08ac007e40 tx=0x7f08ac00e4b0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 shutdown_connections 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08a00683b0 0x7f08a006a870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b40b1f70 0x7f08b40701b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f08b409ae10 0x7f08b4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 --2- 172.21.15.22:0/1185042119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b409aa30 0x7f08b4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 >> 172.21.15.22:0/1185042119 conn(0x7f08b40b4100 msgr2=0x7f08b4071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 shutdown_connections 2024-04-27T07:51:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.038+0000 7f08ba727640 1 -- 172.21.15.22:0/1185042119 wait complete. 2024-04-27T07:51:47.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-27T07:51:47.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 Processor -- start 2024-04-27T07:51:47.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 -- start start 2024-04-27T07:51:47.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f94240b3330 0x7f94240d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94240d17b0 con 0x7f942406dcc0 2024-04-27T07:51:47.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94240d1a50 con 0x7f942409b140 2024-04-27T07:51:47.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942c728640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f94240d1d30 con 0x7f94240b3330 2024-04-27T07:51:47.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942a4c4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f942406dcc0 0x7f942409aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59052/0 (socket says 172.21.15.22:59052) 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942a4c4640 1 -- 172.21.15.22:0/3231772055 learned_addr learned my addr 172.21.15.22:0/3231772055 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 --2- 172.21.15.22:0/3231772055 >> v2:172.21.15.183:3300/0 conn(0x7f94240b3330 0x7f94240d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871546146 0 0) 0x7f94240d17b0 con 0x7f942406dcc0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9410003530 con 0x7f942406dcc0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2530285013 0 0) 0x7f9410003530 con 0x7f942406dcc0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 >> v2:172.21.15.183:3300/0 conn(0x7f94240b3330 msgr2=0x7f94240d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 --2- 172.21.15.22:0/3231772055 >> v2:172.21.15.183:3300/0 conn(0x7f94240b3330 0x7f94240d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 >> v1:172.21.15.183:6789/0 conn(0x7f942409b140 legacy=0x7f94240d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240d2010 con 0x7f942406dcc0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f9420001680 con 0x7f942406dcc0 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 >> v1:172.21.15.22:6789/0 conn(0x7f942406dcc0 legacy=0x7f942409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100039c0 0x7f9410003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9410004810 0x7f9410004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 0x7f941000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94240d1d30 con 0x7f94100051d0 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f941000c170 con 0x7f9410004810 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f941000c370 con 0x7f94100039c0 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 0x7f941000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f9429cc3640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9410004810 0x7f9410004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942a4c4640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100039c0 0x7f9410003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 -- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100039c0 msgr2=0x7f9410003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100039c0 0x7f9410003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 -- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9410004810 msgr2=0x7f9410004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9410004810 0x7f9410004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942acc5640 1 -- 172.21.15.22:0/3231772055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9410003530 con 0x7f94100051d0 2024-04-27T07:51:47.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.194+0000 7f942a4c4640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100039c0 0x7f9410003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942acc5640 1 -- 172.21.15.22:0/3231772055 reap_dead start 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942acc5640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 0x7f941000b9d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fc0 tx=0x7f941c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2541846b4bb3b171 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c009c10 con 0x7f94100051d0 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c0a5cd0 con 0x7f94100051d0 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f94294c2640 1 -- 172.21.15.22:0/3231772055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c003be0 con 0x7f94100051d0 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 -- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 msgr2=0x7f941000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 0x7f941000b9d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fc0 tx=0x7f941c01ccc0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 -- 172.21.15.22:0/3231772055 shutdown_connections 2024-04-27T07:51:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 --2- 172.21.15.22:0/3231772055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94100051d0 0x7f941000b9d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 -- 172.21.15.22:0/3231772055 >> 172.21.15.22:0/3231772055 conn(0x7f94240b4140 msgr2=0x7f94240b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 -- 172.21.15.22:0/3231772055 shutdown_connections 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.198+0000 7f942c728640 1 -- 172.21.15.22:0/3231772055 wait complete. 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 Processor -- start 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- start start 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100051d0 0x7f94240c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 0x7f942415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f942409aa30 con 0x7f94241e90b0 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f942409abb0 con 0x7f94241e9530 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f942409ad30 con 0x7f94100051d0 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45360/0 (socket says 172.21.15.22:45360) 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 -- 172.21.15.22:0/3343911334 learned_addr learned my addr 172.21.15.22:0/3343911334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942a4c4640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100051d0 0x7f94240c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942acc5640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 0x7f942415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 -- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100051d0 msgr2=0x7f94240c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100051d0 0x7f94240c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 -- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 msgr2=0x7f942415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 0x7f942415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 -- 172.21.15.22:0/3343911334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240c69b0 con 0x7f94241e90b0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942acc5640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 0x7f942415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f9429cc3640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f941400e4e0 tx=0x7f941400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140052e0 con 0x7f94241e90b0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9414018070 con 0x7f94241e90b0 2024-04-27T07:51:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94240c6c90 con 0x7f94241e90b0 2024-04-27T07:51:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414005480 con 0x7f94241e90b0 2024-04-27T07:51:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94240c7250 con 0x7f94241e90b0 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942404ff10 con 0x7f94241e90b0 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9414003840 con 0x7f94241e90b0 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.202+0000 7f941affd640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 0x7f940806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.206+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7f94140864e0 con 0x7f94241e90b0 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.206+0000 7f942a4c4640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 0x7f940806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:47.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.206+0000 7f942a4c4640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 0x7f940806a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9420001a20 tx=0x7f9420000fe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:47.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.210+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9414056b80 con 0x7f94241e90b0 2024-04-27T07:51:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:47.370+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94240b4d30 con 0x7f94241e90b0 2024-04-27T07:51:48.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.042+0000 7f941affd640 1 -- 172.21.15.22:0/3343911334 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f9414056480 con 0x7f94241e90b0 2024-04-27T07:51:48.048 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-04-27T07:51:48.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 msgr2=0x7f940806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 0x7f940806a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9420001a20 tx=0x7f9420000fe0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 msgr2=0x7f942415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f941400e4e0 tx=0x7f941400e9a0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 shutdown_connections 2024-04-27T07:51:48.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9408068380 0x7f940806a840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94241e9530 0x7f942415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94241e90b0 0x7f942415e2a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 --2- 172.21.15.22:0/3343911334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94100051d0 0x7f94240c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.046+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 >> 172.21.15.22:0/3343911334 conn(0x7f94240b4140 msgr2=0x7f9424074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:48.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.050+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 shutdown_connections 2024-04-27T07:51:48.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.050+0000 7f942c728640 1 -- 172.21.15.22:0/3343911334 wait complete. 2024-04-27T07:51:48.082 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-27T07:51:48.082 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-27T07:51:48.082 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-27T07:51:48.082 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:48.083 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-27T07:51:48.083 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-27T07:51:48.083 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-27T07:51:48.083 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:48.083 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-27T07:51:48.083 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-27T07:51:48.084 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-27T07:51:48.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:48.084 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-27T07:51:48.084 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-27T07:51:48.084 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-27T07:51:48.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:51:48.093 INFO:tasks.ceph.mds.c:Stopped 2024-04-27T07:51:48.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-27T07:51:48.098 INFO:tasks.ceph.mds.a:Stopped 2024-04-27T07:51:48.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-27T07:51:48.099 INFO:tasks.ceph.mds.b:Stopped 2024-04-27T07:51:48.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-27T07:51:48.100 INFO:tasks.ceph.mds.d:Stopped 2024-04-27T07:51:48.101 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-27T07:51:48.140 INFO:tasks.ceph.mds.c:Started 2024-04-27T07:51:48.140 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:48.141 INFO:tasks.ceph.mds.a:Started 2024-04-27T07:51:48.142 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:48.142 INFO:tasks.ceph.mds.b:Started 2024-04-27T07:51:48.143 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:48.143 INFO:tasks.ceph.mds.d:Started 2024-04-27T07:51:48.143 DEBUG:teuthology.parallel:result is None 2024-04-27T07:51:48.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-27T07:51:48.204 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-04-27T07:51:48.205 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-04-27T07:51:48.212 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2024-04-27T07:51:48.217 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2024-04-27T07:51:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 Processor -- start 2024-04-27T07:51:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 -- start start 2024-04-27T07:51:48.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4f400d0b50 0x7f4f400d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:48.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f40161100 con 0x7f4f400d05a0 2024-04-27T07:51:48.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4009aa30 con 0x7f4f400d0140 2024-04-27T07:51:48.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f47f1d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4009ad10 con 0x7f4f400d0b50 2024-04-27T07:51:48.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f464ba640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4f400d05a0 0x7f4f401609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59072/0 (socket says 172.21.15.22:59072) 2024-04-27T07:51:48.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f464ba640 1 -- 172.21.15.22:0/3615594416 learned_addr learned my addr 172.21.15.22:0/3615594416 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:48.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.266+0000 7f4f45cb9640 1 --2- 172.21.15.22:0/3615594416 >> v2:172.21.15.183:3300/0 conn(0x7f4f400d0b50 0x7f4f400d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065373656 0 0) 0x7f4f40161100 con 0x7f4f400d05a0 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f2c003530 con 0x7f4f400d05a0 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f45cb9640 1 -- 172.21.15.22:0/3615594416 >> v1:172.21.15.183:6789/0 conn(0x7f4f400d0140 legacy=0x7f4f401602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409184085 0 0) 0x7f4f4009aa30 con 0x7f4f400d0140 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f45cb9640 1 -- 172.21.15.22:0/3615594416 >> v1:172.21.15.22:6789/0 conn(0x7f4f400d05a0 legacy=0x7f4f401609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f45cb9640 1 -- 172.21.15.22:0/3615594416 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4009aff0 con 0x7f4f400d0b50 2024-04-27T07:51:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f45cb9640 1 --2- 172.21.15.22:0/3615594416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400d0f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0097c0 tx=0x7f4f3c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a709a4843800f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0a7070 con 0x7f4f400d0b50 2024-04-27T07:51:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3c0b0070 con 0x7f4f400d0b50 2024-04-27T07:51:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f44cb7640 1 -- 172.21.15.22:0/3615594416 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0ab890 con 0x7f4f400d0b50 2024-04-27T07:51:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 -- 172.21.15.22:0/3615594416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 msgr2=0x7f4f400d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/3615594416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400d0f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0097c0 tx=0x7f4f3c01c470 comp rx=0 tx=0).stop 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 -- 172.21.15.22:0/3615594416 shutdown_connections 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/3615594416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400d0f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 -- 172.21.15.22:0/3615594416 >> 172.21.15.22:0/3615594416 conn(0x7f4f400b4060 msgr2=0x7f4f400b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 -- 172.21.15.22:0/3615594416 shutdown_connections 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.270+0000 7f4f47f1d640 1 -- 172.21.15.22:0/3615594416 wait complete. 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 Processor -- start 2024-04-27T07:51:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- start start 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f400d0140 0x7f4f4015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400d1470 con 0x7f4f400d05a0 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4015cb30 con 0x7f4f400d0b50 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f400ce050 con 0x7f4f400d0140 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45392/0 (socket says 172.21.15.22:45392) 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 -- 172.21.15.22:0/679859946 learned_addr learned my addr 172.21.15.22:0/679859946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f464ba640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f45cb9640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f400d0140 0x7f4f4015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 -- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f400d0140 msgr2=0x7f4f4015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f400d0140 0x7f4f4015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 -- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 msgr2=0x7f4f400b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 -- 172.21.15.22:0/679859946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f400b53e0 con 0x7f4f400d05a0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f454b8640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c4d0 tx=0x7f4f3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017070 con 0x7f4f400d05a0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f30005480 con 0x7f4f400d05a0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30003840 con 0x7f4f400d05a0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f400b56c0 con 0x7f4f400d05a0 2024-04-27T07:51:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.274+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f400b5c80 con 0x7f4f400d05a0 2024-04-27T07:51:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40068c30 con 0x7f4f400d05a0 2024-04-27T07:51:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f3001a070 con 0x7f4f400d05a0 2024-04-27T07:51:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f367fc640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 0x7f4f2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f4f30085990 con 0x7f4f400d05a0 2024-04-27T07:51:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f45cb9640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 0x7f4f2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:48.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.278+0000 7f4f45cb9640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 0x7f4f2406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0af000 tx=0x7f4f3c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:48.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.282+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f30055cc0 con 0x7f4f400d05a0 2024-04-27T07:51:48.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:48.458+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4f400b4cc0 con 0x7f4f400d05a0 2024-04-27T07:51:49.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.062+0000 7f4f367fc640 1 -- 172.21.15.22:0/679859946 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f4f30059fc0 con 0x7f4f400d05a0 2024-04-27T07:51:49.070 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-04-27T07:51:49.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 msgr2=0x7f4f2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 0x7f4f2406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0af000 tx=0x7f4f3c0a7000 comp rx=0 tx=0).stop 2024-04-27T07:51:49.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 msgr2=0x7f4f400b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c4d0 tx=0x7f4f3000c990 comp rx=0 tx=0).stop 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 shutdown_connections 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4f24068380 0x7f4f2406a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f400d0b50 0x7f4f400b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f400d05a0 0x7f4f400b6480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 --2- 172.21.15.22:0/679859946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f400d0140 0x7f4f4015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 >> 172.21.15.22:0/679859946 conn(0x7f4f400b4060 msgr2=0x7f4f40153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 shutdown_connections 2024-04-27T07:51:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.066+0000 7f4f47f1d640 1 -- 172.21.15.22:0/679859946 wait complete. 2024-04-27T07:51:49.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-27T07:51:49.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 Processor -- start 2024-04-27T07:51:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 -- start start 2024-04-27T07:51:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f709c09b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f709c09aa30 0x7f709c09aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c0d0670 con 0x7f709c0d0190 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0d0950 con 0x7f709c09aa30 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f70a0ca0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f709c0cd300 con 0x7f709c15c850 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f7099d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f709c0d0190 0x7f709c0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59076/0 (socket says 172.21.15.22:59076) 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f7099d9b640 1 -- 172.21.15.22:0/1682312926 learned_addr learned my addr 172.21.15.22:0/1682312926 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f709c09b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1682312926 >> v2:172.21.15.183:3300/0 conn(0x7f709c09aa30 0x7f709c09aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932849230 0 0) 0x7f709c0d0670 con 0x7f709c0d0190 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7084003530 con 0x7f709c0d0190 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2483857664 0 0) 0x7f7084003530 con 0x7f709c0d0190 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 msgr2=0x7f709c09b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f709c09b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 >> v2:172.21.15.183:3300/0 conn(0x7f709c09aa30 msgr2=0x7f709c09aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 --2- 172.21.15.22:0/1682312926 >> v2:172.21.15.183:3300/0 conn(0x7f709c09aa30 0x7f709c09aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0cd5e0 con 0x7f709c0d0190 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7090001680 con 0x7f709c0d0190 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 >> v1:172.21.15.22:6789/0 conn(0x7f709c0d0190 legacy=0x7f709c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70840039c0 0x7f7084003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7084004810 0x7f7084004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f70840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0d0950 con 0x7f70840039c0 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0cd300 con 0x7f7084004810 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f708400b9d0 con 0x7f709c15c850 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70840039c0 0x7f7084003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f7099d9b640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7084004810 0x7f7084004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f70840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 -- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 msgr2=0x7f70840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f70840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 -- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7084004810 msgr2=0x7f7084004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7084004810 0x7f7084004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709a59c640 1 -- 172.21.15.22:0/1682312926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7084003530 con 0x7f70840039c0 2024-04-27T07:51:49.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f7099d9b640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7084004810 0x7f7084004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.222+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c15c850 0x7f70840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f709a59c640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70840039c0 0x7f7084003de0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f708c009ae0 tx=0x7f708c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ac9c97a0081c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0096b0 con 0x7f70840039c0 2024-04-27T07:51:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708c0a8440 con 0x7f70840039c0 2024-04-27T07:51:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f709959a640 1 -- 172.21.15.22:0/1682312926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0a8830 con 0x7f70840039c0 2024-04-27T07:51:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1682312926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70840039c0 msgr2=0x7f7084003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1682312926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70840039c0 0x7f7084003de0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f708c009ae0 tx=0x7f708c01ca10 comp rx=0 tx=0).stop 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f709ad9d640 1 -- 172.21.15.22:0/1682312926 reap_dead start 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1682312926 shutdown_connections 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1682312926 >> 172.21.15.22:0/1682312926 conn(0x7f709c0b40b0 msgr2=0x7f709c0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1682312926 shutdown_connections 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1682312926 wait complete. 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 Processor -- start 2024-04-27T07:51:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.226+0000 7f70a0ca0640 1 -- start start 2024-04-27T07:51:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 0x7f709c0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c0b7ea0 0x7f709c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0b2010 con 0x7f709c15a2c0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0b1e90 con 0x7f70840039c0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f709c0b2190 con 0x7f709c0b7ea0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45418/0 (socket says 172.21.15.22:45418) 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 -- 172.21.15.22:0/1942666503 learned_addr learned my addr 172.21.15.22:0/1942666503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f7099d9b640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c0b7ea0 0x7f709c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709a59c640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 0x7f709c0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 -- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c0b7ea0 msgr2=0x7f709c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c0b7ea0 0x7f709c15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 -- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 msgr2=0x7f709c0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 0x7f709c0c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 -- 172.21.15.22:0/1942666503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0c7820 con 0x7f709c15a2c0 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709a59c640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 0x7f709c0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709ad9d640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f709c072ba0 tx=0x7f709400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709401c070 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7094017400 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c15dd90 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70940177d0 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c0c7540 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709c069340 con 0x7f709c15a2c0 2024-04-27T07:51:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f709401f070 con 0x7f709c15a2c0 2024-04-27T07:51:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 0x7f707c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7094086650 con 0x7f709c15a2c0 2024-04-27T07:51:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.230+0000 7f709a59c640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 0x7f707c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.234+0000 7f709a59c640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 0x7f707c06a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f708c0af000 tx=0x7f708c0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.234+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f709c15dd90 con 0x7f709c15a2c0 2024-04-27T07:51:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.458+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f709c0cc980 con 0x7f709c15a2c0 2024-04-27T07:51:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.458+0000 7f708affd640 1 -- 172.21.15.22:0/1942666503 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f709c0c7820 con 0x7f709c15a2c0 2024-04-27T07:51:49.465 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:49.465 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAmrixmAjl3OxAA96NFo3XfB80lw+5T7bIoGw==", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-04-27T07:51:49.466 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixm3NCuDhAA08t9OL34T+b7oRLKgRT8FA==", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmPrNtCBAA2fbFI8ZI7gmXDQnmz6YHDw==", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-04-27T07:51:49.467 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmWz6DFBAARdqqspUPofRg+F8n27WfPQ==", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.468 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAorixmHG63LBAAE19Tgew8foVNVEgoA4GTrQ==", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQArrixmcnNOMBAAbRIY/UbPaMEI8JZvxbP8Lg==", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.469 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAurixm4v/5NhAAzHclJ5aq4/QO7MPYYoYxKA==", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAyrixmMuufAhAAwKgeVoF0TXe1gLsDKVw4LA==", 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.470 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQA4rixm8Yw6AhAAeUHuDVW0FWW5Iohg+iOxNA==", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.471 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQA7rixmuWyjBhAAFDNZDS+j+WOczPvGKkuZ1Q==", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQA+rixm5imsDRAAo0sn0MkBlRLIvTnCmhGXbQ==", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBBrixmgO8CFBAAOJPLxZIn/68YBTgx+Z6igA==", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmLWgVGhAAj3agIuftac+O2Tov9zpCZQ==", 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.473 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmPaU9IhAAvwk0hgdanfEl/bY5dgaRow==", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmawBhHxAArTddaaaUap4fpOevEXznJQ==", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAnrixmKKl4JxAAgYg/XgMW2yJ6/5PKmiGgDA==", 2024-04-27T07:51:49.475 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAmrixmaWM0JBAAb/2DeNmyAcgAupnC1hDySA==", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.476 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mds", 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixmoMXJFBAAvOf89wZGM/yUbVxW9gC6Bg==", 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mds" 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mgr", 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixmVQDKFBAAtbscEpJG+8JPBWeVvXb25A==", 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-27T07:51:49.477 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-osd", 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixmQDHKFBAABUhqrBzE47e5OdAwrilTCA==", 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-osd" 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd", 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixm4mrKFBAA2bmqt1YmQsVCp8ucLs+xIw==", 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-27T07:51:49.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixmE5zKFBAAHwo/5T0G8OKIAjlNc3tLAQ==", 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rgw", 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBHrixmdM7KFBAAvXGIVY9gMV4aiYcJMr2jzA==", 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-27T07:51:49.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAmrixm8NccOBAA5JurAK+VIdg5PQtbTzIfLg==", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQAmrixmXgdLNRAAjhEHY9rn1R6e6o/YiIxO5A==", 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-04-27T07:51:49.480 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-27T07:51:49.481 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 msgr2=0x7f707c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 0x7f707c06a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f708c0af000 tx=0x7f708c0b2040 comp rx=0 tx=0).stop 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 msgr2=0x7f709c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f709c072ba0 tx=0x7f709400f4a0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 shutdown_connections 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f707c068380 0x7f707c06a840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f709c15a2c0 0x7f709c15a740 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f709c0b7ea0 0x7f709c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 --2- 172.21.15.22:0/1942666503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70840039c0 0x7f709c0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 >> 172.21.15.22:0/1942666503 conn(0x7f709c0b40b0 msgr2=0x7f709c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 shutdown_connections 2024-04-27T07:51:49.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.462+0000 7f70a0ca0640 1 -- 172.21.15.22:0/1942666503 wait complete. 2024-04-27T07:51:49.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-27T07:51:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 Processor -- start 2024-04-27T07:51:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 -- start start 2024-04-27T07:51:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6e9415ebe0 0x7f6e940cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6e940b7930 0x7f6e940d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e940b3d30 con 0x7f6e940d4790 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6e940d4540 con 0x7f6e9415ebe0 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.598+0000 7f6e9b0ad640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6e940ba7a0 con 0x7f6e940b7930 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e98e49640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6e940d4790 0x7f6e940d4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59092/0 (socket says 172.21.15.22:59092) 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e98e49640 1 -- 172.21.15.22:0/1874199578 learned_addr learned my addr 172.21.15.22:0/1874199578 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/1874199578 >> v2:172.21.15.22:3300/0 conn(0x7f6e9415ebe0 0x7f6e940cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9964a640 1 --2- 172.21.15.22:0/1874199578 >> v2:172.21.15.183:3300/0 conn(0x7f6e940b7930 0x7f6e940d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8b7fe640 1 -- 172.21.15.22:0/1874199578 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569299097 0 0) 0x7f6e90001120 con 0x7f6e940d4790 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8b7fe640 1 -- 172.21.15.22:0/1874199578 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e7c003530 con 0x7f6e940d4790 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 -- 172.21.15.22:0/1874199578 >> v2:172.21.15.183:3300/0 conn(0x7f6e940b7930 msgr2=0x7f6e940d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/1874199578 >> v2:172.21.15.183:3300/0 conn(0x7f6e940b7930 0x7f6e940d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 -- 172.21.15.22:0/1874199578 >> v1:172.21.15.22:6789/0 conn(0x7f6e940d4790 legacy=0x7f6e940d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 -- 172.21.15.22:0/1874199578 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e940ba4c0 con 0x7f6e9415ebe0 2024-04-27T07:51:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/1874199578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e9415ebe0 0x7f6e940cf860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6e940b39b0 tx=0x7f6e8401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10704162b1e8a07c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8b7fe640 1 -- 172.21.15.22:0/1874199578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840b1070 con 0x7f6e9415ebe0 2024-04-27T07:51:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8b7fe640 1 -- 172.21.15.22:0/1874199578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e840ac620 con 0x7f6e9415ebe0 2024-04-27T07:51:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e8b7fe640 1 -- 172.21.15.22:0/1874199578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840ac9f0 con 0x7f6e9415ebe0 2024-04-27T07:51:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/1874199578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e9415ebe0 msgr2=0x7f6e940cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/1874199578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e9415ebe0 0x7f6e940cf860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6e940b39b0 tx=0x7f6e8401c3c0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/1874199578 shutdown_connections 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/1874199578 >> v2:172.21.15.183:3300/0 conn(0x7f6e940b7930 0x7f6e940d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/1874199578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e9415ebe0 0x7f6e940cf860 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/1874199578 >> 172.21.15.22:0/1874199578 conn(0x7f6e94067d10 msgr2=0x7f6e94068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/1874199578 shutdown_connections 2024-04-27T07:51:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.602+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/1874199578 wait complete. 2024-04-27T07:51:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 Processor -- start 2024-04-27T07:51:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- start start 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 0x7f6e9415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 0x7f6e94159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940cff20 con 0x7f6e940d4790 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940cfda0 con 0x7f6e940ccf10 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940d3500 con 0x7f6e940b7930 2024-04-27T07:51:49.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45428/0 (socket says 172.21.15.22:45428) 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 -- 172.21.15.22:0/2608030195 learned_addr learned my addr 172.21.15.22:0/2608030195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9964a640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 0x7f6e94159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 -- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 msgr2=0x7f6e9415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e98e49640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 0x7f6e9415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 0x7f6e9415aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 -- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 msgr2=0x7f6e94159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 0x7f6e94159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 -- 172.21.15.22:0/2608030195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e94149f50 con 0x7f6e940d4790 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9964a640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 0x7f6e94159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e98e49640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 0x7f6e9415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e8bfff640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6e84019ff0 tx=0x7f6e84019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840b1040 con 0x7f6e940d4790 2024-04-27T07:51:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e840ba070 con 0x7f6e940d4790 2024-04-27T07:51:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9414a230 con 0x7f6e940d4790 2024-04-27T07:51:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840b5650 con 0x7f6e940d4790 2024-04-27T07:51:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9414a7f0 con 0x7f6e940d4790 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.606+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e94068c30 con 0x7f6e940d4790 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.610+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6e840b57f0 con 0x7f6e940d4790 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.610+0000 7f6e897fa640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 0x7f6e8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.610+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f6e841203d0 con 0x7f6e940d4790 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.610+0000 7f6e98e49640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 0x7f6e8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.610+0000 7f6e98e49640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 0x7f6e8006a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e900012c0 tx=0x7f6e90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.614+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e84016070 con 0x7f6e940d4790 2024-04-27T07:51:49.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.838+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f6e940b8a20 con 0x7f6e940d4790 2024-04-27T07:51:49.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e897fa640 1 -- 172.21.15.22:0/2608030195 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f6e840f0cd0 con 0x7f6e940d4790 2024-04-27T07:51:49.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 msgr2=0x7f6e8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 0x7f6e8006a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e900012c0 tx=0x7f6e90009040 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 msgr2=0x7f6e94149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6e84019ff0 tx=0x7f6e84019160 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 shutdown_connections 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6e800683b0 0x7f6e8006a870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e940ccf10 0x7f6e94159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e940d4790 0x7f6e94149850 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 --2- 172.21.15.22:0/2608030195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e940b7930 0x7f6e9415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 >> 172.21.15.22:0/2608030195 conn(0x7f6e94067d10 msgr2=0x7f6e9415ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 shutdown_connections 2024-04-27T07:51:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.842+0000 7f6e9b0ad640 1 -- 172.21.15.22:0/2608030195 wait complete. 2024-04-27T07:51:49.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-27T07:51:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.970+0000 7fd2b9430640 1 Processor -- start 2024-04-27T07:51:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.970+0000 7fd2b9430640 1 -- start start 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b9430640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd2b4144e40 0x7fd2b406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b9430640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 0x7fd2b406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b9430640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b4145220 con 0x7fd2b406ef10 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b9430640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b4162df0 con 0x7fd2b4144e40 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b9430640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b41630d0 con 0x7fd2b406f560 2024-04-27T07:51:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b27fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd2b406ef10 0x7fd2b41623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50442/0 (socket says 172.21.15.22:50442) 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b27fc640 1 -- 172.21.15.22:0/2284022134 learned_addr learned my addr 172.21.15.22:0/2284022134 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 0x7fd2b406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/2284022134 >> v2:172.21.15.183:3300/0 conn(0x7fd2b4144e40 0x7fd2b406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3220778534 0 0) 0x7fd2a8001120 con 0x7fd2b406ef10 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd29c003530 con 0x7fd2b406ef10 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3812825266 0 0) 0x7fd29c003530 con 0x7fd2b406ef10 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 msgr2=0x7fd2b406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 0x7fd2b406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 >> v2:172.21.15.183:3300/0 conn(0x7fd2b4144e40 msgr2=0x7fd2b406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 --2- 172.21.15.22:0/2284022134 >> v2:172.21.15.183:3300/0 conn(0x7fd2b4144e40 0x7fd2b406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b406e1d0 con 0x7fd2b406ef10 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fd2a8001850 con 0x7fd2b406ef10 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 >> v1:172.21.15.22:6789/0 conn(0x7fd2b406ef10 legacy=0x7fd2b41623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd29c0039c0 0x7fd29c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 0x7fd29c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd29c004d50 0x7fd29c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b4162df0 con 0x7fd29c0039c0 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b41630d0 con 0x7fd29c004d50 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd29c00b9d0 con 0x7fd2b406f560 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd29c0039c0 0x7fd29c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd29c004d50 0x7fd29c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 msgr2=0x7fd29c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b406f560 0x7fd29c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd29c004d50 msgr2=0x7fd29c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd29c004d50 0x7fd29c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/2284022134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c003530 con 0x7fd29c0039c0 2024-04-27T07:51:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.974+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd29c004d50 0x7fd29c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd29c0039c0 0x7fd29c003de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009ae0 tx=0x7fd2a401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdc123e4af5bd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40b2070 con 0x7fd29c0039c0 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a40a5610 con 0x7fd29c0039c0 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b1ffb640 1 -- 172.21.15.22:0/2284022134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40096b0 con 0x7fd29c0039c0 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- 172.21.15.22:0/2284022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd29c0039c0 msgr2=0x7fd29c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 --2- 172.21.15.22:0/2284022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd29c0039c0 0x7fd29c003de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009ae0 tx=0x7fd2a401cab0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b37fe640 1 -- 172.21.15.22:0/2284022134 reap_dead start 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- 172.21.15.22:0/2284022134 shutdown_connections 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- 172.21.15.22:0/2284022134 >> 172.21.15.22:0/2284022134 conn(0x7fd2b4067f10 msgr2=0x7fd2b406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- 172.21.15.22:0/2284022134 shutdown_connections 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- 172.21.15.22:0/2284022134 wait complete. 2024-04-27T07:51:49.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 Processor -- start 2024-04-27T07:51:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- start start 2024-04-27T07:51:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b41f7750 0x7fd2b41fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 0x7fd2b41634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b406fed0 con 0x7fd2b4163a10 2024-04-27T07:51:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b406fd50 con 0x7fd2b414c940 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b9430640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b40af460 con 0x7fd2b41f7750 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37594/0 (socket says 172.21.15.22:37594) 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.978+0000 7fd2b37fe640 1 -- 172.21.15.22:0/411790743 learned_addr learned my addr 172.21.15.22:0/411790743 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b41f7750 0x7fd2b41fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b27fc640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 0x7fd2b41634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 -- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b41f7750 msgr2=0x7fd2b41fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b41f7750 0x7fd2b41fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 -- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 msgr2=0x7fd2b41634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 0x7fd2b41634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 -- 172.21.15.22:0/411790743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b41fc2e0 con 0x7fd2b4163a10 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b27fc640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 0x7fd2b41634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b37fe640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40702d0 tx=0x7fd2ac00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac01c070 con 0x7fd2b4163a10 2024-04-27T07:51:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00fe90 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b414cee0 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac017620 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b414d4a0 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b4068c30 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd2ac01f070 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 0x7fd2a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd2ac0866a0 con 0x7fd2b4163a10 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 0x7fd2a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.982+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 0x7fd2a006a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40af000 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:49.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:49.986+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2ac012070 con 0x7fd2b4163a10 2024-04-27T07:51:50.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.210+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd2b41620c0 con 0x7fd2b4163a10 2024-04-27T07:51:50.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.214+0000 7fd29b7fe640 1 -- 172.21.15.22:0/411790743 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fd2ac056fa0 con 0x7fd2b4163a10 2024-04-27T07:51:50.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 msgr2=0x7fd2a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 0x7fd2a006a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40af000 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 msgr2=0x7fd2b41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40702d0 tx=0x7fd2ac00f4a0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 shutdown_connections 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd2a0068380 0x7fd2a006a840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2b4163a10 0x7fd2b41402e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2b414c940 0x7fd2b41634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 --2- 172.21.15.22:0/411790743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2b41f7750 0x7fd2b41fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 >> 172.21.15.22:0/411790743 conn(0x7fd2b4067f10 msgr2=0x7fd2b4142440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 shutdown_connections 2024-04-27T07:51:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.218+0000 7fd2b9430640 1 -- 172.21.15.22:0/411790743 wait complete. 2024-04-27T07:51:50.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-27T07:51:50.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 Processor -- start 2024-04-27T07:51:50.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 -- start start 2024-04-27T07:51:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f091c0d1760 0x7f091c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091c15fac0 con 0x7f091c15ef40 2024-04-27T07:51:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091c15fda0 con 0x7f091c15c780 2024-04-27T07:51:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f09211e0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f091c162790 con 0x7f091c0d1760 2024-04-27T07:51:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f091c15ef40 0x7f091c0d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50454/0 (socket says 172.21.15.22:50454) 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 -- 172.21.15.22:0/272649352 learned_addr learned my addr 172.21.15.22:0/272649352 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091b59e640 1 --2- 172.21.15.22:0/272649352 >> v2:172.21.15.183:3300/0 conn(0x7f091c0d1760 0x7f091c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036667257 0 0) 0x7f091c15fac0 con 0x7f091c15ef40 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0908003530 con 0x7f091c15ef40 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3936459432 0 0) 0x7f0908003530 con 0x7f091c15ef40 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 >> v2:172.21.15.183:3300/0 conn(0x7f091c0d1760 msgr2=0x7f091c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 --2- 172.21.15.22:0/272649352 >> v2:172.21.15.183:3300/0 conn(0x7f091c0d1760 0x7f091c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 >> v1:172.21.15.183:6789/0 conn(0x7f091c15c780 legacy=0x7f091c162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c0ba760 con 0x7f091c15ef40 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0904001680 con 0x7f091c15ef40 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 >> v1:172.21.15.22:6789/0 conn(0x7f091c15ef40 legacy=0x7f091c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09080039c0 0x7f0908003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f0908004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09080052f0 0x7f090800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f091c162790 con 0x7f0908004810 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f090800c1c0 con 0x7f09080052f0 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f090800c3c0 con 0x7f09080039c0 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f0908004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091b59e640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09080052f0 0x7f090800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 -- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09080039c0 msgr2=0x7f0908003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091ad9d640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09080039c0 0x7f0908003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09080039c0 0x7f0908003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 -- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09080052f0 msgr2=0x7f090800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09080052f0 0x7f090800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f0913fff640 1 -- 172.21.15.22:0/272649352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0908003530 con 0x7f0908004810 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091b59e640 1 -- 172.21.15.22:0/272649352 reap_dead start 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.358+0000 7f091ad9d640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09080039c0 0x7f0908003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f0913fff640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f0908004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0904001ec0 tx=0x7f090401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc480837c913538 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904007d40 con 0x7f0908004810 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09040a9070 con 0x7f0908004810 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f091a59c640 1 -- 172.21.15.22:0/272649352 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904005d80 con 0x7f0908004810 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 msgr2=0x7f0908004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f0908004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0904001ec0 tx=0x7f090401cc20 comp rx=0 tx=0).stop 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- 172.21.15.22:0/272649352 shutdown_connections 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 --2- 172.21.15.22:0/272649352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f0908004c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- 172.21.15.22:0/272649352 >> 172.21.15.22:0/272649352 conn(0x7f091c067910 msgr2=0x7f091c067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- 172.21.15.22:0/272649352 shutdown_connections 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- 172.21.15.22:0/272649352 wait complete. 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 Processor -- start 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- start start 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091c0c8420 0x7f091c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f091c0b8460 0x7f091c0b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f091c0d3610 con 0x7f0908004810 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f091c0d3490 con 0x7f091c0c8420 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f09211e0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f091c162f70 con 0x7f091c0b8460 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f091ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.362+0000 7f091ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37612/0 (socket says 172.21.15.22:37612) 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 -- 172.21.15.22:0/1902139058 learned_addr learned my addr 172.21.15.22:0/1902139058 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091b59e640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f091c0b8460 0x7f091c0b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f0913fff640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091c0c8420 0x7f091c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 -- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f091c0b8460 msgr2=0x7f091c0b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f091c0b8460 0x7f091c0b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 -- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091c0c8420 msgr2=0x7f091c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091c0c8420 0x7f091c0cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 -- 172.21.15.22:0/1902139058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c0cb310 con 0x7f0908004810 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f091ad9d640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d4a0 tx=0x7f090c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c019070 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c0054a0 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c007270 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c0c89c0 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c0c8f80 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c068c30 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f090c007410 con 0x7f0908004810 2024-04-27T07:51:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 0x7f08fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f090c085bc0 con 0x7f0908004810 2024-04-27T07:51:50.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.366+0000 7f0913fff640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 0x7f08fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.370+0000 7f0913fff640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 0x7f08fc06a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09040ab000 tx=0x7f09040a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.370+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f090c056620 con 0x7f0908004810 2024-04-27T07:51:50.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.582+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f091c0d2590 con 0x7f0908004810 2024-04-27T07:51:50.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.586+0000 7f09137fe640 1 -- 172.21.15.22:0/1902139058 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f090c055f20 con 0x7f0908004810 2024-04-27T07:51:50.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 msgr2=0x7f08fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 0x7f08fc06a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09040ab000 tx=0x7f09040a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:50.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 msgr2=0x7f091c14fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d4a0 tx=0x7f090c00d960 comp rx=0 tx=0).stop 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 shutdown_connections 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f08fc068380 0x7f08fc06a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f091c0b8460 0x7f091c0b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091c0c8420 0x7f091c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 --2- 172.21.15.22:0/1902139058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0908004810 0x7f091c14fec0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 >> 172.21.15.22:0/1902139058 conn(0x7f091c067910 msgr2=0x7f091c0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:50.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 shutdown_connections 2024-04-27T07:51:50.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.590+0000 7f09211e0640 1 -- 172.21.15.22:0/1902139058 wait complete. 2024-04-27T07:51:50.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-27T07:51:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.730+0000 7fee53fe3640 1 Processor -- start 2024-04-27T07:51:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.730+0000 7fee53fe3640 1 -- start start 2024-04-27T07:51:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee53fe3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee53fe3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fee4c145a00 0x7fee4c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee53fe3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4c06e050 con 0x7fee4c072030 2024-04-27T07:51:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee53fe3640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fee4c06f330 con 0x7fee4c145a00 2024-04-27T07:51:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee53fe3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c0ceae0 con 0x7fee4c1454b0 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee5157e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fee4c145a00 0x7fee4c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:57764/0 (socket says 172.21.15.22:57764) 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 -- 172.21.15.22:0/3191823459 learned_addr learned my addr 172.21.15.22:0/3191823459 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 -- 172.21.15.22:0/3191823459 >> v2:172.21.15.183:3300/0 conn(0x7fee4c145a00 msgr2=0x7fee4c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 --2- 172.21.15.22:0/3191823459 >> v2:172.21.15.183:3300/0 conn(0x7fee4c145a00 0x7fee4c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 -- 172.21.15.22:0/3191823459 >> v1:172.21.15.183:6789/0 conn(0x7fee4c072030 legacy=0x7fee4c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.734+0000 7fee51d7f640 1 -- 172.21.15.22:0/3191823459 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c0cedc0 con 0x7fee4c1454b0 2024-04-27T07:51:50.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee51d7f640 1 --2- 172.21.15.22:0/3191823459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009e50 tx=0x7fee3c01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b8716c575b4f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee50d7d640 1 -- 172.21.15.22:0/3191823459 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0a5700 con 0x7fee4c1454b0 2024-04-27T07:51:50.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee50d7d640 1 -- 172.21.15.22:0/3191823459 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3c0a58a0 con 0x7fee4c1454b0 2024-04-27T07:51:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee50d7d640 1 -- 172.21.15.22:0/3191823459 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0a5c70 con 0x7fee4c1454b0 2024-04-27T07:51:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- 172.21.15.22:0/3191823459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 msgr2=0x7fee4c072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- 172.21.15.22:0/3191823459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009e50 tx=0x7fee3c01ca30 comp rx=0 tx=0).stop 2024-04-27T07:51:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- 172.21.15.22:0/3191823459 shutdown_connections 2024-04-27T07:51:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- 172.21.15.22:0/3191823459 >> v2:172.21.15.183:3300/0 conn(0x7fee4c145a00 0x7fee4c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- 172.21.15.22:0/3191823459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c1454b0 0x7fee4c072fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- 172.21.15.22:0/3191823459 >> 172.21.15.22:0/3191823459 conn(0x7fee4c067f10 msgr2=0x7fee4c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- 172.21.15.22:0/3191823459 shutdown_connections 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- 172.21.15.22:0/3191823459 wait complete. 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 Processor -- start 2024-04-27T07:51:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- start start 2024-04-27T07:51:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c072030 0x7fee4c13c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee4c145a00 0x7fee4c0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c072850 con 0x7fee4c1454b0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c15d820 con 0x7fee4c145a00 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee53fe3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c15fa30 con 0x7fee4c072030 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37620/0 (socket says 172.21.15.22:37620) 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 -- 172.21.15.22:0/2372838747 learned_addr learned my addr 172.21.15.22:0/2372838747 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee52580640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee4c145a00 0x7fee4c0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee51d7f640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c072030 0x7fee4c13c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 -- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c072030 msgr2=0x7fee4c13c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c072030 0x7fee4c13c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 -- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee4c145a00 msgr2=0x7fee4c0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee4c145a00 0x7fee4c0b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.738+0000 7fee5157e640 1 -- 172.21.15.22:0/2372838747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c14ae10 con 0x7fee4c1454b0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee5157e640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fee4000e930 tx=0x7fee4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40017070 con 0x7fee4c1454b0 2024-04-27T07:51:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4000ca00 con 0x7fee4c1454b0 2024-04-27T07:51:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4000cdb0 con 0x7fee4c1454b0 2024-04-27T07:51:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c14b0f0 con 0x7fee4c1454b0 2024-04-27T07:51:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c14b6b0 con 0x7fee4c1454b0 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c068c30 con 0x7fee4c1454b0 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fee4001a070 con 0x7fee4c1454b0 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 0x7fee3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fee400868e0 con 0x7fee4c1454b0 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee51d7f640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 0x7fee3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.742+0000 7fee51d7f640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 0x7fee3806a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0ae000 tx=0x7fee3c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.746+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee40056cf0 con 0x7fee4c1454b0 2024-04-27T07:51:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.970+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fee4c0b0f60 con 0x7fee4c1454b0 2024-04-27T07:51:50.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.970+0000 7fee4a7fc640 1 -- 172.21.15.22:0/2372838747 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fee4005aff0 con 0x7fee4c1454b0 2024-04-27T07:51:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 msgr2=0x7fee3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 0x7fee3806a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0ae000 tx=0x7fee3c0a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 msgr2=0x7fee4c0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fee4000e930 tx=0x7fee4000edf0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 shutdown_connections 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee38068380 0x7fee3806a840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee4c145a00 0x7fee4c0b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee4c1454b0 0x7fee4c0b1d10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 --2- 172.21.15.22:0/2372838747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee4c072030 0x7fee4c13c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 >> 172.21.15.22:0/2372838747 conn(0x7fee4c067f10 msgr2=0x7fee4c072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 shutdown_connections 2024-04-27T07:51:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:50.974+0000 7fee53fe3640 1 -- 172.21.15.22:0/2372838747 wait complete. 2024-04-27T07:51:50.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-27T07:51:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 Processor -- start 2024-04-27T07:51:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 -- start start 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a941660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a940720d0 con 0x7f4a9406f870 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a94072370 con 0x7f4a94144dd0 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a98a9c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4a94072650 con 0x7f4a940cfe10 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a91d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4a9406f870 0x7f4a940cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50466/0 (socket says 172.21.15.22:50466) 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a91d9b640 1 -- 172.21.15.22:0/1532180860 learned_addr learned my addr 172.21.15.22:0/1532180860 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a941660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911362078 0 0) 0x7f4a940720d0 con 0x7f4a9406f870 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a7c005740 con 0x7f4a9406f870 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1466562100 0 0) 0x7f4a7c005740 con 0x7f4a9406f870 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 msgr2=0x7f4a941660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a941660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 >> v1:172.21.15.183:6789/0 conn(0x7f4a94144dd0 legacy=0x7f4a9406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a94072930 con 0x7f4a9406f870 2024-04-27T07:51:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f4a880016a0 con 0x7f4a9406f870 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a941660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 >> v1:172.21.15.22:6789/0 conn(0x7f4a9406f870 legacy=0x7f4a940cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c005c00 0x7f4a7c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a7c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a7c006f60 0x7f4a7c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94072650 con 0x7f4a7c006f60 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c00dcf0 con 0x7f4a7c005c00 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c00def0 con 0x7f4a940cfe10 2024-04-27T07:51:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a7c006f60 0x7f4a7c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9259c640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c005c00 0x7f4a7c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a91d9b640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a7c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 -- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 msgr2=0x7f4a7c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940cfe10 0x7f4a7c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 -- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c005c00 msgr2=0x7f4a7c006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c005c00 0x7f4a7c006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a92d9d640 1 -- 172.21.15.22:0/1532180860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c005740 con 0x7f4a7c006f60 2024-04-27T07:51:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.110+0000 7f4a9259c640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c005c00 0x7f4a7c006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a7c006f60 0x7f4a7c0073e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0073b0 tx=0x7f4a8c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a614f100bb0e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0b2070 con 0x7f4a7c006f60 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a5dc0 con 0x7f4a7c006f60 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a9159a640 1 -- 172.21.15.22:0/1532180860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009c80 con 0x7f4a7c006f60 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- 172.21.15.22:0/1532180860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a7c006f60 msgr2=0x7f4a7c0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/1532180860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a7c006f60 0x7f4a7c0073e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0073b0 tx=0x7f4a8c01cec0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a92d9d640 1 -- 172.21.15.22:0/1532180860 reap_dead start 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- 172.21.15.22:0/1532180860 shutdown_connections 2024-04-27T07:51:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- 172.21.15.22:0/1532180860 >> 172.21.15.22:0/1532180860 conn(0x7f4a94067f10 msgr2=0x7f4a9406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:51.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- 172.21.15.22:0/1532180860 shutdown_connections 2024-04-27T07:51:51.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- 172.21.15.22:0/1532180860 wait complete. 2024-04-27T07:51:51.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 Processor -- start 2024-04-27T07:51:51.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- start start 2024-04-27T07:51:51.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 0x7f4a94143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940ac4f0 0x7f4a940cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4a940b4de0 con 0x7f4a940ac970 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941451b0 con 0x7f4a7c006f60 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a98a9c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4a940b4f60 con 0x7f4a940ac4f0 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.114+0000 7f4a92d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37644/0 (socket says 172.21.15.22:37644) 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 -- 172.21.15.22:0/2249023700 learned_addr learned my addr 172.21.15.22:0/2249023700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:51.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a9259c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 0x7f4a94143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a91d9b640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940ac4f0 0x7f4a940cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 -- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940ac4f0 msgr2=0x7f4a940cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940ac4f0 0x7f4a940cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 -- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 msgr2=0x7f4a94143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 0x7f4a94143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 -- 172.21.15.22:0/2249023700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a941f4f30 con 0x7f4a940ac970 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a9259c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 0x7f4a94143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a92d9d640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c002240 tx=0x7f4a7c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0b2040 con 0x7f4a940ac970 2024-04-27T07:51:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a5bc0 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009c80 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a940c97e0 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a940c9a80 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a94144270 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4a8c0adce0 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 0x7f4a7406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f4a8c120960 con 0x7f4a940ac970 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.118+0000 7f4a9259c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 0x7f4a7406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.122+0000 7f4a9259c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 0x7f4a7406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a84007fd0 tx=0x7f4a84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.122+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a8c016bd0 con 0x7f4a940ac970 2024-04-27T07:51:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.338+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f4a94068c30 con 0x7f4a940ac970 2024-04-27T07:51:51.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a82ffd640 1 -- 172.21.15.22:0/2249023700 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f4a8c0f0c00 con 0x7f4a940ac970 2024-04-27T07:51:51.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 msgr2=0x7f4a7406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 0x7f4a7406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a84007fd0 tx=0x7f4a84009040 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 msgr2=0x7f4a940cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c002240 tx=0x7f4a7c007a50 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 shutdown_connections 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4a74068380 0x7f4a7406a840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a940ac970 0x7f4a940cb760 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a940ac4f0 0x7f4a940cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 --2- 172.21.15.22:0/2249023700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7c006f60 0x7f4a94143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 >> 172.21.15.22:0/2249023700 conn(0x7f4a94067f10 msgr2=0x7f4a940cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 shutdown_connections 2024-04-27T07:51:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.342+0000 7f4a98a9c640 1 -- 172.21.15.22:0/2249023700 wait complete. 2024-04-27T07:51:51.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-27T07:51:51.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 Processor -- start 2024-04-27T07:51:51.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 -- start start 2024-04-27T07:51:51.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f424415b880 0x7f42440cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42440b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 0x7f42440b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f42440ccbf0 con 0x7f424415b880 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f424415bc60 con 0x7f42440b4810 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424c2b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f42440cc020 con 0x7f42440ce6d0 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f424415b880 0x7f42440cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f424415b880 0x7f42440cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37656/0 (socket says 172.21.15.22:37656) 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 -- 172.21.15.22:0/3404782773 learned_addr learned my addr 172.21.15.22:0/3404782773 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a84f640 1 --2- 172.21.15.22:0/3404782773 >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 0x7f42440b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424984d640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42440b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 -- 172.21.15.22:0/3404782773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 msgr2=0x7f42440b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42440b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 -- 172.21.15.22:0/3404782773 >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 msgr2=0x7f42440b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 --2- 172.21.15.22:0/3404782773 >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 0x7f42440b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a04e640 1 -- 172.21.15.22:0/3404782773 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440cc2c0 con 0x7f424415b880 2024-04-27T07:51:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.474+0000 7f424a84f640 1 --2- 172.21.15.22:0/3404782773 >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 0x7f42440b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f424415b880 0x7f42440cd080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42440b2f80 tx=0x7f424001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f73826f933f499 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424904c640 1 -- 172.21.15.22:0/3404782773 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400b1070 con 0x7f424415b880 2024-04-27T07:51:51.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424904c640 1 -- 172.21.15.22:0/3404782773 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42400ac630 con 0x7f424415b880 2024-04-27T07:51:51.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424904c640 1 -- 172.21.15.22:0/3404782773 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400a8050 con 0x7f424415b880 2024-04-27T07:51:51.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- 172.21.15.22:0/3404782773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f424415b880 msgr2=0x7f42440cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f424415b880 0x7f42440cd080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42440b2f80 tx=0x7f424001c520 comp rx=0 tx=0).stop 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- 172.21.15.22:0/3404782773 shutdown_connections 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- 172.21.15.22:0/3404782773 >> v2:172.21.15.183:3300/0 conn(0x7f42440b4810 0x7f42440b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42440b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- 172.21.15.22:0/3404782773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f424415b880 0x7f42440cd080 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- 172.21.15.22:0/3404782773 >> 172.21.15.22:0/3404782773 conn(0x7f42440b2fb0 msgr2=0x7f42440b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- 172.21.15.22:0/3404782773 shutdown_connections 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- 172.21.15.22:0/3404782773 wait complete. 2024-04-27T07:51:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 Processor -- start 2024-04-27T07:51:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- start start 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42441478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 0x7f424415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f424415fb70 con 0x7f42440b4810 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42440cecc0 con 0x7f424415b880 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424c2b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f424415fcf0 con 0x7f42440ce6d0 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37668/0 (socket says 172.21.15.22:37668) 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 -- 172.21.15.22:0/1900748807 learned_addr learned my addr 172.21.15.22:0/1900748807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424984d640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42441478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a84f640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 0x7f424415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 -- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 msgr2=0x7f42441478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42441478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 -- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 msgr2=0x7f424415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 0x7f424415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a04e640 1 -- 172.21.15.22:0/1900748807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4244155b30 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.478+0000 7f424a84f640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 0x7f424415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424a04e640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42400097c0 tx=0x7f42400192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400bc070 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42400b7430 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4244155e10 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400b1040 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441645c0 con 0x7f42440b4810 2024-04-27T07:51:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244069340 con 0x7f42440b4810 2024-04-27T07:51:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f42400bf070 con 0x7f42440b4810 2024-04-27T07:51:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 0x7f422806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f424011fcb0 con 0x7f42440b4810 2024-04-27T07:51:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424984d640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 0x7f422806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.482+0000 7f424984d640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 0x7f422806a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f42440cd2a0 tx=0x7f4234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.486+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4240016110 con 0x7f42440b4810 2024-04-27T07:51:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.698+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f42440b7400 con 0x7f42440b4810 2024-04-27T07:51:51.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.702+0000 7f423affd640 1 -- 172.21.15.22:0/1900748807 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f42400f05c0 con 0x7f42440b4810 2024-04-27T07:51:51.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 msgr2=0x7f422806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 0x7f422806a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f42440cd2a0 tx=0x7f4234008040 comp rx=0 tx=0).stop 2024-04-27T07:51:51.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 msgr2=0x7f4244155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42400097c0 tx=0x7f42400192c0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 shutdown_connections 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4228068380 0x7f422806a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f424415b880 0x7f424415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42440ce6d0 0x7f42441478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 --2- 172.21.15.22:0/1900748807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42440b4810 0x7f4244155400 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 >> 172.21.15.22:0/1900748807 conn(0x7f42440b2fb0 msgr2=0x7f424406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 shutdown_connections 2024-04-27T07:51:51.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.706+0000 7f424c2b2640 1 -- 172.21.15.22:0/1900748807 wait complete. 2024-04-27T07:51:51.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:51.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.838+0000 7f09b2368640 1 Processor -- start 2024-04-27T07:51:51.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 -- start start 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09ac06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f09ac0ce260 0x7f09ac0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ac06e630 con 0x7f09ac162da0 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f09ac163180 con 0x7f09ac0ce260 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b2368640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac06f1b0 con 0x7f09ac06fc70 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09abfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f09ac162da0 0x7f09ac06f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50478/0 (socket says 172.21.15.22:50478) 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09abfff640 1 -- 172.21.15.22:0/1674929910 learned_addr learned my addr 172.21.15.22:0/1674929910 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09ac06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b0905640 1 --2- 172.21.15.22:0/1674929910 >> v2:172.21.15.183:3300/0 conn(0x7f09ac0ce260 0x7f09ac0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642636108 0 0) 0x7f09ac06e630 con 0x7f09ac162da0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0994003530 con 0x7f09ac162da0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1279197948 0 0) 0x7f0994003530 con 0x7f09ac162da0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 msgr2=0x7f09ac06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09ac06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 >> v2:172.21.15.183:3300/0 conn(0x7f09ac0ce260 msgr2=0x7f09ac0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 --2- 172.21.15.22:0/1674929910 >> v2:172.21.15.183:3300/0 conn(0x7f09ac0ce260 0x7f09ac0b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac06e910 con 0x7f09ac162da0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f099c001680 con 0x7f09ac162da0 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 >> v1:172.21.15.22:6789/0 conn(0x7f09ac162da0 legacy=0x7f09ac06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09940039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0994003f40 0x7f0994004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994004d50 0x7f09940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac06f1b0 con 0x7f0994003f40 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f099400b9d0 con 0x7f0994004d50 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f099400bb50 con 0x7f09ac06fc70 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0994003f40 0x7f0994004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b0905640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994004d50 0x7f09940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09abfff640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09940039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 -- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 msgr2=0x7f09940039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09940039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 -- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994004d50 msgr2=0x7f09940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994004d50 0x7f09940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09ab7fe640 1 -- 172.21.15.22:0/1674929910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0994003530 con 0x7f0994003f40 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09b0905640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994004d50 0x7f09940051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.842+0000 7f09abfff640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac06fc70 0x7f09940039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:51.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0994003f40 0x7f0994004810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ab0 tx=0x7f09a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c686b7a2530264c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00b2070 con 0x7f0994003f40 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a00a5640 con 0x7f0994003f40 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09aaffd640 1 -- 172.21.15.22:0/1674929910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00096b0 con 0x7f0994003f40 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- 172.21.15.22:0/1674929910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0994003f40 msgr2=0x7f0994004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 --2- 172.21.15.22:0/1674929910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0994003f40 0x7f0994004810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ab0 tx=0x7f09a001c9f0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b0905640 1 -- 172.21.15.22:0/1674929910 reap_dead start 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- 172.21.15.22:0/1674929910 shutdown_connections 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- 172.21.15.22:0/1674929910 >> 172.21.15.22:0/1674929910 conn(0x7f09ac067f10 msgr2=0x7f09ac06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- 172.21.15.22:0/1674929910 shutdown_connections 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- 172.21.15.22:0/1674929910 wait complete. 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 Processor -- start 2024-04-27T07:51:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- start start 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 0x7f09ac14c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 0x7f09ac160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac06eed0 con 0x7f09ac14d140 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac0b43b0 con 0x7f0994003f40 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b2368640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac06e050 con 0x7f09ac15fce0 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37676/0 (socket says 172.21.15.22:37676) 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 -- 172.21.15.22:0/2073541875 learned_addr learned my addr 172.21.15.22:0/2073541875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b0905640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 0x7f09ac160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09abfff640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 0x7f09ac14c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 -- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 msgr2=0x7f09ac160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 0x7f09ac160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 -- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 msgr2=0x7f09ac14c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 0x7f09ac14c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09ab7fe640 1 -- 172.21.15.22:0/2073541875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac14ce30 con 0x7f09ac14d140 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09b0905640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 0x7f09ac160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.846+0000 7f09abfff640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 0x7f09ac14c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09ab7fe640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f09a001c820 tx=0x7f0994005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00b2040 con 0x7f09ac14d140 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a00a5e10 con 0x7f09ac14d140 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0004020 con 0x7f09ac14d140 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac1625a0 con 0x7f09ac14d140 2024-04-27T07:51:51.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1622c0 con 0x7f09ac14d140 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac068c30 con 0x7f09ac14d140 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09a00c0030 con 0x7f09ac14d140 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 0x7f099806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f09a01202b0 con 0x7f09ac14d140 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09abfff640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 0x7f099806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:51.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.850+0000 7f09abfff640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 0x7f099806a870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f099c001210 tx=0x7f09ac06e800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:51.858+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09a00c7060 con 0x7f09ac14d140 2024-04-27T07:51:52.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.018+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09ac0b3fe0 con 0x7f09ac14d140 2024-04-27T07:51:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09a8ff9640 1 -- 172.21.15.22:0/2073541875 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12867 (secure 0 0 0) 0x7f09a0016760 con 0x7f09ac14d140 2024-04-27T07:51:52.028 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:52.028 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:49.078723+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 msgr2=0x7f099806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 0x7f099806a870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f099c001210 tx=0x7f09ac06e800 comp rx=0 tx=0).stop 2024-04-27T07:51:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 msgr2=0x7f09ac15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f09a001c820 tx=0x7f0994005840 comp rx=0 tx=0).stop 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 shutdown_connections 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f09980683b0 0x7f099806a870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09ac15fce0 0x7f09ac160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09ac14d140 0x7f09ac15d410 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 --2- 172.21.15.22:0/2073541875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0994003f40 0x7f09ac14c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 >> 172.21.15.22:0/2073541875 conn(0x7f09ac067f10 msgr2=0x7f09ac0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 shutdown_connections 2024-04-27T07:51:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.022+0000 7f09b2368640 1 -- 172.21.15.22:0/2073541875 wait complete. 2024-04-27T07:51:52.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:51:52.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:51:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 Processor -- start 2024-04-27T07:51:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 -- start start 2024-04-27T07:51:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2fb80cfed0 0x7f2fb80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb815d410 con 0x7f2fb806f0b0 2024-04-27T07:51:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb815ccc0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbf84a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb815e020 con 0x7f2fb815b5e0 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:57858/0 (socket says 172.21.15.22:57858) 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 -- 172.21.15.22:0/1290937771 learned_addr learned my addr 172.21.15.22:0/1290937771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbdde7640 1 --2- 172.21.15.22:0/1290937771 >> v2:172.21.15.183:3300/0 conn(0x7f2fb80cfed0 0x7f2fb80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 -- 172.21.15.22:0/1290937771 >> v2:172.21.15.183:3300/0 conn(0x7f2fb80cfed0 msgr2=0x7f2fb80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 --2- 172.21.15.22:0/1290937771 >> v2:172.21.15.183:3300/0 conn(0x7f2fb80cfed0 0x7f2fb80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 -- 172.21.15.22:0/1290937771 >> v1:172.21.15.183:6789/0 conn(0x7f2fb806f0b0 legacy=0x7f2fb80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.166+0000 7f2fbd5e6640 1 -- 172.21.15.22:0/1290937771 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb815d6f0 con 0x7f2fb815b5e0 2024-04-27T07:51:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbd5e6640 1 --2- 172.21.15.22:0/1290937771 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009870 tx=0x7f2fac01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c0af97eb90e960 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fabfff640 1 -- 172.21.15.22:0/1290937771 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a95f0 con 0x7f2fb815b5e0 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fabfff640 1 -- 172.21.15.22:0/1290937771 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0a9790 con 0x7f2fb815b5e0 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fabfff640 1 -- 172.21.15.22:0/1290937771 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a9b60 con 0x7f2fb815b5e0 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1290937771 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 msgr2=0x7f2fb815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1290937771 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009870 tx=0x7f2fac01c530 comp rx=0 tx=0).stop 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1290937771 shutdown_connections 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1290937771 >> v2:172.21.15.183:3300/0 conn(0x7f2fb80cfed0 0x7f2fb80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1290937771 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb815c780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1290937771 >> 172.21.15.22:0/1290937771 conn(0x7f2fb809aa30 msgr2=0x7f2fb80d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1290937771 shutdown_connections 2024-04-27T07:51:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1290937771 wait complete. 2024-04-27T07:51:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 Processor -- start 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.170+0000 7f2fbf84a640 1 -- start start 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fb806f0b0 0x7f2fb80b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb81561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb815d1f0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb815d070 con 0x7f2fb806f0b0 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb80d1df0 con 0x7f2fb815b5e0 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37682/0 (socket says 172.21.15.22:37682) 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 -- 172.21.15.22:0/1743152230 learned_addr learned my addr 172.21.15.22:0/1743152230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbdde7640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb81561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbd5e6640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fb806f0b0 0x7f2fb80b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 -- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 msgr2=0x7f2fb81561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb81561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 -- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fb806f0b0 msgr2=0x7f2fb80b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fb806f0b0 0x7f2fb80b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 -- 172.21.15.22:0/1743152230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb80b2780 con 0x7f2fb80cfed0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbcde5640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c790 tx=0x7f2fb400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4016020 con 0x7f2fb80cfed0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb4006d60 con 0x7f2fb80cfed0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb80b24a0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4003bc0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb80c9bc0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb804ff10 con 0x7f2fb80cfed0 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2fb400a850 con 0x7f2fb80cfed0 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 0x7f2f9c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2fb40863e0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.174+0000 7f2fbd5e6640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 0x7f2f9c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.178+0000 7f2fbd5e6640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 0x7f2f9c06a870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ae000 tx=0x7f2fac0b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.178+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2fb4020020 con 0x7f2fb80cfed0 2024-04-27T07:51:52.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fb815c540 con 0x7f2fb80cfed0 2024-04-27T07:51:52.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fa9ffb640 1 -- 172.21.15.22:0/1743152230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f2fb4056ce0 con 0x7f2fb80cfed0 2024-04-27T07:51:52.345 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:51:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 msgr2=0x7f2f9c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 0x7f2f9c06a870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ae000 tx=0x7f2fac0b3040 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 msgr2=0x7f2fb814c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c790 tx=0x7f2fb400cc50 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 shutdown_connections 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2f9c0683b0 0x7f2f9c06a870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fb815b5e0 0x7f2fb81561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fb80cfed0 0x7f2fb814c6b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 --2- 172.21.15.22:0/1743152230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fb806f0b0 0x7f2fb80b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 >> 172.21.15.22:0/1743152230 conn(0x7f2fb809aa30 msgr2=0x7f2fb8070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 shutdown_connections 2024-04-27T07:51:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.338+0000 7f2fbf84a640 1 -- 172.21.15.22:0/1743152230 wait complete. 2024-04-27T07:51:52.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-27T07:51:52.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 Processor -- start 2024-04-27T07:51:52.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 -- start start 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a30144dd0 0x7f7a3006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a3006f6a0 0x7f7a300cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a300720d0 con 0x7f7a300cfc40 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7a3006fb00 con 0x7f7a3006f6a0 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a37432640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a30072370 con 0x7f7a30144dd0 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a3006f6a0 0x7f7a300cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a351ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a30144dd0 0x7f7a3006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a3006f6a0 0x7f7a300cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:47350/0 (socket says 172.21.15.22:47350) 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 -- 172.21.15.22:0/497335164 learned_addr learned my addr 172.21.15.22:0/497335164 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a359cf640 1 --1- 172.21.15.22:0/497335164 >> v1:172.21.15.183:6789/0 conn(0x7f7a300cfc40 0x7f7a3015baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:53442/0 (socket says 172.21.15.22:53442) 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 -- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a30144dd0 msgr2=0x7f7a3006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 --2- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a30144dd0 0x7f7a3006f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 -- 172.21.15.22:0/497335164 >> v1:172.21.15.183:6789/0 conn(0x7f7a300cfc40 legacy=0x7f7a3015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.478+0000 7f7a349cd640 1 -- 172.21.15.22:0/497335164 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a30072650 con 0x7f7a3006f6a0 2024-04-27T07:51:52.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a349cd640 1 --2- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a3006f6a0 0x7f7a300cf700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009870 tx=0x7f7a2c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6f05d0bc141fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a27fff640 1 -- 172.21.15.22:0/497335164 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0ab650 con 0x7f7a3006f6a0 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a27fff640 1 -- 172.21.15.22:0/497335164 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c0b0070 con 0x7f7a3006f6a0 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a27fff640 1 -- 172.21.15.22:0/497335164 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0aba10 con 0x7f7a3006f6a0 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a3006f6a0 msgr2=0x7f7a300cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 --2- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a3006f6a0 0x7f7a300cf700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009870 tx=0x7f7a2c01c560 comp rx=0 tx=0).stop 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- 172.21.15.22:0/497335164 shutdown_connections 2024-04-27T07:51:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 --2- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a3006f6a0 0x7f7a300cf700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 --2- 172.21.15.22:0/497335164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a30144dd0 0x7f7a3006f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- 172.21.15.22:0/497335164 >> 172.21.15.22:0/497335164 conn(0x7f7a30067f10 msgr2=0x7f7a3006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- 172.21.15.22:0/497335164 shutdown_connections 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- 172.21.15.22:0/497335164 wait complete. 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 Processor -- start 2024-04-27T07:51:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.482+0000 7f7a37432640 1 -- start start 2024-04-27T07:51:52.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a300cfc40 0x7f7a301592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a30144dd0 0x7f7a3013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a300d01d0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a300d0050 con 0x7f7a30144dd0 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a300b4de0 con 0x7f7a300cfc40 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37684/0 (socket says 172.21.15.22:37684) 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 -- 172.21.15.22:0/546889321 learned_addr learned my addr 172.21.15.22:0/546889321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a359cf640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a30144dd0 0x7f7a3013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a349cd640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a300cfc40 0x7f7a301592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 -- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a300cfc40 msgr2=0x7f7a301592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a300cfc40 0x7f7a301592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 -- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a30144dd0 msgr2=0x7f7a3013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a30144dd0 0x7f7a3013ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 -- 172.21.15.22:0/546889321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3013ccf0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a351ce640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000e8a0 tx=0x7f7a2000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20017070 con 0x7f7a3006f6a0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2000c940 con 0x7f7a3006f6a0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a30159d10 con 0x7f7a3006f6a0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000ccf0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301f5430 con 0x7f7a3006f6a0 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a30068c30 con 0x7f7a3006f6a0 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7a2001a070 con 0x7f7a3006f6a0 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 0x7f7a1406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7a20086450 con 0x7f7a3006f6a0 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.486+0000 7f7a349cd640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 0x7f7a1406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.490+0000 7f7a349cd640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 0x7f7a1406a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af000 tx=0x7f7a2c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.490+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a20056df0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7a300b58c0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a25ffb640 1 -- 172.21.15.22:0/546889321 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7f7a200566f0 con 0x7f7a3006f6a0 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:51:52.660 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-04-27T07:51:52.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 msgr2=0x7f7a1406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 0x7f7a1406a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af000 tx=0x7f7a2c0a7000 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 msgr2=0x7f7a30138f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000e8a0 tx=0x7f7a2000eca0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 shutdown_connections 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7a140683b0 0x7f7a1406a870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a30144dd0 0x7f7a3013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a300cfc40 0x7f7a301592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 --2- 172.21.15.22:0/546889321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3006f6a0 0x7f7a30138f30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 >> 172.21.15.22:0/546889321 conn(0x7f7a30067f10 msgr2=0x7f7a3013c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 shutdown_connections 2024-04-27T07:51:52.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.654+0000 7f7a37432640 1 -- 172.21.15.22:0/546889321 wait complete. 2024-04-27T07:51:52.678 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-27T07:51:52.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-27T07:51:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.786+0000 7ff9e3c62640 1 Processor -- start 2024-04-27T07:51:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.786+0000 7ff9e3c62640 1 -- start start 2024-04-27T07:51:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e3c62640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff9dc071f70 0x7ff9dc072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e3c62640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc072890 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e3c62640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc0cdb10 con 0x7ff9dc06ee50 2024-04-27T07:51:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e3c62640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc0ce0d0 con 0x7ff9dc071f70 2024-04-27T07:51:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e11fd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff9dc0cffe0 0x7ff9dc06e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50484/0 (socket says 172.21.15.22:50484) 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e11fd640 1 -- 172.21.15.22:0/3655406686 learned_addr learned my addr 172.21.15.22:0/3655406686 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/3655406686 >> v2:172.21.15.183:3300/0 conn(0x7ff9dc071f70 0x7ff9dc072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764692486 0 0) 0x7ff9dc072890 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c8003530 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 494924891 0 0) 0x7ff9c8003530 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 >> v2:172.21.15.183:3300/0 conn(0x7ff9dc071f70 msgr2=0x7ff9dc072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 --2- 172.21.15.22:0/3655406686 >> v2:172.21.15.183:3300/0 conn(0x7ff9dc071f70 0x7ff9dc072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 >> v1:172.21.15.183:6789/0 conn(0x7ff9dc06ee50 legacy=0x7ff9dc06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0cddf0 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7ff9cc001680 con 0x7ff9dc0cffe0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 >> v1:172.21.15.22:6789/0 conn(0x7ff9dc0cffe0 legacy=0x7ff9dc06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80039c0 0x7ff9c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c8004810 0x7ff9c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 0x7ff9c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0ce0d0 con 0x7ff9c80051d0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c800c170 con 0x7ff9c8004810 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c800c370 con 0x7ff9c80039c0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 0x7ff9c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80039c0 0x7ff9c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e11fd640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c8004810 0x7ff9c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 -- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80039c0 msgr2=0x7ff9c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80039c0 0x7ff9c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 -- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c8004810 msgr2=0x7ff9c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c8004810 0x7ff9c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 -- 172.21.15.22:0/3655406686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c8003530 con 0x7ff9c80051d0 2024-04-27T07:51:52.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80039c0 0x7ff9c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:52.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e11fd640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c8004810 0x7ff9c8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.790+0000 7ff9e21ff640 1 -- 172.21.15.22:0/3655406686 reap_dead start 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 0x7ff9c800b9d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4001780 tx=0x7ff9d401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94977fa07a4571a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40013a0 con 0x7ff9c80051d0 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40a9070 con 0x7ff9c80051d0 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e09fc640 1 -- 172.21.15.22:0/3655406686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4003600 con 0x7ff9c80051d0 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 msgr2=0x7ff9c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 0x7ff9c800b9d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4001780 tx=0x7ff9d401c420 comp rx=0 tx=0).stop 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/3655406686 shutdown_connections 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/3655406686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c80051d0 0x7ff9c800b9d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/3655406686 >> 172.21.15.22:0/3655406686 conn(0x7ff9dc0b21d0 msgr2=0x7ff9dc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/3655406686 shutdown_connections 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/3655406686 wait complete. 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 Processor -- start 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- start start 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80051d0 0x7ff9dc0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9dc15f260 0x7ff9dc1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc06dfd0 con 0x7ff9dc1e6060 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0cd8c0 con 0x7ff9dc15f260 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc146470 con 0x7ff9c80051d0 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37698/0 (socket says 172.21.15.22:37698) 2024-04-27T07:51:52.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 -- 172.21.15.22:0/847888452 learned_addr learned my addr 172.21.15.22:0/847888452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80051d0 0x7ff9dc0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e11fd640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9dc15f260 0x7ff9dc1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 -- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80051d0 msgr2=0x7ff9dc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80051d0 0x7ff9dc0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 -- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9dc15f260 msgr2=0x7ff9dc1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9dc15f260 0x7ff9dc1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 -- 172.21.15.22:0/847888452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc141c80 con 0x7ff9dc1e6060 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e21ff640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff9d401ccc0 tx=0x7ff9c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bb070 con 0x7ff9dc1e6060 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40050d0 con 0x7ff9dc1e6060 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0ad9d0 con 0x7ff9dc1e6060 2024-04-27T07:51:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40013a0 con 0x7ff9dc1e6060 2024-04-27T07:51:52.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.794+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0adf90 con 0x7ff9dc1e6060 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc04ff10 con 0x7ff9dc1e6060 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff9d40be070 con 0x7ff9dc1e6060 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9d27fc640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 0x7ff9c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff9d411f220 con 0x7ff9dc1e6060 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 0x7ff9c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:52.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.798+0000 7ff9e19fe640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 0x7ff9c006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b5610 tx=0x7ff9d800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.802+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9d4016130 con 0x7ff9dc1e6060 2024-04-27T07:51:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:52.966+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9dc072cb0 con 0x7ff9dc1e6060 2024-04-27T07:51:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.082+0000 7ff9d27fc640 1 -- 172.21.15.22:0/847888452 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7ff9d40efbc0 con 0x7ff9dc1e6060 2024-04-27T07:51:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 msgr2=0x7ff9c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 0x7ff9c006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b5610 tx=0x7ff9d800b040 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 msgr2=0x7ff9dc1e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff9d401ccc0 tx=0x7ff9c8005840 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 shutdown_connections 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff9c00683b0 0x7ff9c006a870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9dc1e6060 0x7ff9dc1e64e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9dc15f260 0x7ff9dc1411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 --2- 172.21.15.22:0/847888452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9c80051d0 0x7ff9dc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 >> 172.21.15.22:0/847888452 conn(0x7ff9dc0b21d0 msgr2=0x7ff9dc06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 shutdown_connections 2024-04-27T07:51:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.086+0000 7ff9e3c62640 1 -- 172.21.15.22:0/847888452 wait complete. 2024-04-27T07:51:53.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-27T07:51:53.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 Processor -- start 2024-04-27T07:51:53.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 -- start start 2024-04-27T07:51:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee941620c0 0x7fee940d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee940cf2e0 con 0x7fee941624a0 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee940cf580 con 0x7fee940d26d0 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee9980d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fee940b3ae0 con 0x7fee941620c0 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee927fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fee941624a0 0x7fee940d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50498/0 (socket says 172.21.15.22:50498) 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee92ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee941620c0 0x7fee940d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee927fc640 1 -- 172.21.15.22:0/3642985225 learned_addr learned my addr 172.21.15.22:0/3642985225 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee92ffd640 1 -- 172.21.15.22:0/3642985225 >> v1:172.21.15.183:6789/0 conn(0x7fee940d26d0 legacy=0x7fee940d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee92ffd640 1 -- 172.21.15.22:0/3642985225 >> v1:172.21.15.22:6789/0 conn(0x7fee941624a0 legacy=0x7fee940d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.242+0000 7fee92ffd640 1 -- 172.21.15.22:0/3642985225 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940cf860 con 0x7fee941620c0 2024-04-27T07:51:53.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee92ffd640 1 --2- 172.21.15.22:0/3642985225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940d52a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0097c0 tx=0x7fee8c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00b26e3d8234fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee91ffb640 1 -- 172.21.15.22:0/3642985225 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0b0070 con 0x7fee941620c0 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee91ffb640 1 -- 172.21.15.22:0/3642985225 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c0ab600 con 0x7fee941620c0 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee91ffb640 1 -- 172.21.15.22:0/3642985225 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0ab9d0 con 0x7fee941620c0 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 -- 172.21.15.22:0/3642985225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 msgr2=0x7fee940d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 --2- 172.21.15.22:0/3642985225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940d52a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0097c0 tx=0x7fee8c01c460 comp rx=0 tx=0).stop 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 -- 172.21.15.22:0/3642985225 shutdown_connections 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 --2- 172.21.15.22:0/3642985225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940d52a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 -- 172.21.15.22:0/3642985225 >> 172.21.15.22:0/3642985225 conn(0x7fee9408bfc0 msgr2=0x7fee940d47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 -- 172.21.15.22:0/3642985225 shutdown_connections 2024-04-27T07:51:53.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 -- 172.21.15.22:0/3642985225 wait complete. 2024-04-27T07:51:53.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.246+0000 7fee9980d640 1 Processor -- start 2024-04-27T07:51:53.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- start start 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 0x7fee940cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 0x7fee9409f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fee9416cc80 con 0x7fee941620c0 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee940b4ac0 con 0x7fee940d26d0 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fee9416ce00 con 0x7fee941624a0 2024-04-27T07:51:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37716/0 (socket says 172.21.15.22:37716) 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 -- 172.21.15.22:0/3528375853 learned_addr learned my addr 172.21.15.22:0/3528375853 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee92ffd640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 0x7fee940cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee937fe640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 0x7fee9409f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 -- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 msgr2=0x7fee9409f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 0x7fee9409f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 -- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 msgr2=0x7fee940cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 0x7fee940cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 -- 172.21.15.22:0/3528375853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940b84a0 con 0x7fee941620c0 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee937fe640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 0x7fee9409f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee92ffd640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 0x7fee940cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee927fc640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fee8800ca70 tx=0x7fee8800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:53.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88017070 con 0x7fee941620c0 2024-04-27T07:51:53.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8800abf0 con 0x7fee941620c0 2024-04-27T07:51:53.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88012400 con 0x7fee941620c0 2024-04-27T07:51:53.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee940b8780 con 0x7fee941620c0 2024-04-27T07:51:53.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee940b8d40 con 0x7fee941620c0 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9404ff10 con 0x7fee941620c0 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fee8801a070 con 0x7fee941620c0 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.250+0000 7fee7bfff640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 0x7fee8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.254+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7fee88086850 con 0x7fee941620c0 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.254+0000 7fee92ffd640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 0x7fee8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.254+0000 7fee92ffd640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 0x7fee8006a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0ad000 tx=0x7fee8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:53.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.254+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee88021020 con 0x7fee941620c0 2024-04-27T07:51:53.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:53.418+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee940d1140 con 0x7fee941620c0 2024-04-27T07:51:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.090+0000 7fee7bfff640 1 -- 172.21.15.22:0/3528375853 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7fee880570a0 con 0x7fee941620c0 2024-04-27T07:51:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-04-27T07:51:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.090+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 msgr2=0x7fee8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.090+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 0x7fee8006a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0ad000 tx=0x7fee8c008040 comp rx=0 tx=0).stop 2024-04-27T07:51:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.090+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 msgr2=0x7fee940b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.090+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fee8800ca70 tx=0x7fee8800cf30 comp rx=0 tx=0).stop 2024-04-27T07:51:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 shutdown_connections 2024-04-27T07:51:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fee800683b0 0x7fee8006a870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fee941624a0 0x7fee9409f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee941620c0 0x7fee940b7d70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 --2- 172.21.15.22:0/3528375853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee940d26d0 0x7fee940cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.094+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 >> 172.21.15.22:0/3528375853 conn(0x7fee9408bfc0 msgr2=0x7fee9415a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.098+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 shutdown_connections 2024-04-27T07:51:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.098+0000 7fee9980d640 1 -- 172.21.15.22:0/3528375853 wait complete. 2024-04-27T07:51:54.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-27T07:51:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 Processor -- start 2024-04-27T07:51:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 -- start start 2024-04-27T07:51:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f701415ea00 0x7f70140ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70140b9060 con 0x7f70140d1690 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70140b9340 con 0x7f70140d1ad0 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7018e94640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f70140b9be0 con 0x7f701415ea00 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f701259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f701415ea00 0x7f70140ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7011d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f70140d1690 0x7f70140d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50508/0 (socket says 172.21.15.22:50508) 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f7011d9b640 1 -- 172.21.15.22:0/3486274420 learned_addr learned my addr 172.21.15.22:0/3486274420 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f701259c640 1 -- 172.21.15.22:0/3486274420 >> v1:172.21.15.183:6789/0 conn(0x7f70140d1ad0 legacy=0x7f70140b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f701259c640 1 -- 172.21.15.22:0/3486274420 >> v1:172.21.15.22:6789/0 conn(0x7f70140d1690 legacy=0x7f70140d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.258+0000 7f701259c640 1 -- 172.21.15.22:0/3486274420 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70140b9620 con 0x7f701415ea00 2024-04-27T07:51:54.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f701259c640 1 --2- 172.21.15.22:0/3486274420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f701415ea00 0x7f70140ba290 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70040097c0 tx=0x7f700401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cffc3ea6359ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:54.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f701159a640 1 -- 172.21.15.22:0/3486274420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040b0070 con 0x7f701415ea00 2024-04-27T07:51:54.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f701159a640 1 -- 172.21.15.22:0/3486274420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70040ab600 con 0x7f701415ea00 2024-04-27T07:51:54.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f701159a640 1 -- 172.21.15.22:0/3486274420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040ab9d0 con 0x7f701415ea00 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- 172.21.15.22:0/3486274420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f701415ea00 msgr2=0x7f70140ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 --2- 172.21.15.22:0/3486274420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f701415ea00 0x7f70140ba290 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70040097c0 tx=0x7f700401c460 comp rx=0 tx=0).stop 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- 172.21.15.22:0/3486274420 shutdown_connections 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 --2- 172.21.15.22:0/3486274420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f701415ea00 0x7f70140ba290 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- 172.21.15.22:0/3486274420 >> 172.21.15.22:0/3486274420 conn(0x7f7014067d10 msgr2=0x7f7014068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- 172.21.15.22:0/3486274420 shutdown_connections 2024-04-27T07:51:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- 172.21.15.22:0/3486274420 wait complete. 2024-04-27T07:51:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 Processor -- start 2024-04-27T07:51:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.262+0000 7f7018e94640 1 -- start start 2024-04-27T07:51:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70140d1ad0 0x7f7014152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f701415ea00 0x7f7014152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70140d3e10 con 0x7f70140d1690 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70140d3c90 con 0x7f70140d1ad0 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f701416c450 con 0x7f701415ea00 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37734/0 (socket says 172.21.15.22:37734) 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 -- 172.21.15.22:0/3744509134 learned_addr learned my addr 172.21.15.22:0/3744509134 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7011d9b640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70140d1ad0 0x7f7014152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7012d9d640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f701415ea00 0x7f7014152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 -- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f701415ea00 msgr2=0x7f7014152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f701415ea00 0x7f7014152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 -- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70140d1ad0 msgr2=0x7f7014152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70140d1ad0 0x7f7014152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 -- 172.21.15.22:0/3744509134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70140c8880 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f701259c640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70040ae040 tx=0x7f700401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040b0050 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70040bc070 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040b5a50 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141578a0 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70140cb030 con 0x7f70140d1690 2024-04-27T07:51:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7014157690 con 0x7f70140d1690 2024-04-27T07:51:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70040ab4a0 con 0x7f70140d1690 2024-04-27T07:51:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 0x7f6ff406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f7004121d10 con 0x7f70140d1690 2024-04-27T07:51:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.266+0000 7f7011d9b640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 0x7f6ff406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.270+0000 7f7011d9b640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 0x7f6ff406a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7014159e10 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.270+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7004016050 con 0x7f70140d1690 2024-04-27T07:51:54.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:54.474+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f701404ff10 con 0x7f70140d1690 2024-04-27T07:51:55.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.146+0000 7f700affd640 1 -- 172.21.15.22:0/3744509134 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f70040f2410 con 0x7f70140d1690 2024-04-27T07:51:55.155 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 4 and data pool 5 2024-04-27T07:51:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.154+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 msgr2=0x7f6ff406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.154+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 0x7f6ff406a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7014159e10 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-04-27T07:51:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.154+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 msgr2=0x7f70140caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.154+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70040ae040 tx=0x7f700401aca0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 shutdown_connections 2024-04-27T07:51:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6ff4068380 0x7f6ff406a840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f701415ea00 0x7f7014152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70140d1ad0 0x7f7014152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 --2- 172.21.15.22:0/3744509134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70140d1690 0x7f70140caaf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.162+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 >> 172.21.15.22:0/3744509134 conn(0x7f7014067d10 msgr2=0x7f70140d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.166+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 shutdown_connections 2024-04-27T07:51:55.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.166+0000 7f7018e94640 1 -- 172.21.15.22:0/3744509134 wait complete. 2024-04-27T07:51:55.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:55.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.306+0000 7f2159bbc640 1 Processor -- start 2024-04-27T07:51:55.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.306+0000 7f2159bbc640 1 -- start start 2024-04-27T07:51:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2159bbc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f21540d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2159bbc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f21540b7930 0x7f21540d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2159bbc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21540cff70 con 0x7f215415ebe0 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2159bbc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f21540d4540 con 0x7f21540b7930 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2159bbc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f21540b3d30 con 0x7f21540d4790 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2152ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f215415ebe0 0x7f21540cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50512/0 (socket says 172.21.15.22:50512) 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2152ffd640 1 -- 172.21.15.22:0/1903656335 learned_addr learned my addr 172.21.15.22:0/1903656335 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2153fff640 1 --2- 172.21.15.22:0/1903656335 >> v2:172.21.15.183:3300/0 conn(0x7f21540b7930 0x7f21540d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f21540d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808841942 0 0) 0x7f21540cff70 con 0x7f215415ebe0 2024-04-27T07:51:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c003530 con 0x7f215415ebe0 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 439876113 0 0) 0x7f213c003530 con 0x7f215415ebe0 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 msgr2=0x7f21540d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f21540d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 >> v2:172.21.15.183:3300/0 conn(0x7f21540b7930 msgr2=0x7f21540d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 --2- 172.21.15.22:0/1903656335 >> v2:172.21.15.183:3300/0 conn(0x7f21540b7930 0x7f21540d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21540ba7a0 con 0x7f215415ebe0 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f21480016a0 con 0x7f215415ebe0 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 >> v1:172.21.15.22:6789/0 conn(0x7f215415ebe0 legacy=0x7f21540cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f213c0039c0 0x7f213c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f213c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213c004d50 0x7f213c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21540d4540 con 0x7f213c0039c0 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21540b3d30 con 0x7f213c004d50 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f213c00b9d0 con 0x7f21540d4790 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f213c0039c0 0x7f213c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2152ffd640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f213c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f2153fff640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213c004d50 0x7f213c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 -- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 msgr2=0x7f213c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540d4790 0x7f213c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 -- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213c004d50 msgr2=0x7f213c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213c004d50 0x7f213c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 -- 172.21.15.22:0/1903656335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c003530 con 0x7f213c0039c0 2024-04-27T07:51:55.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.310+0000 7f21537fe640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f213c0039c0 0x7f213c003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fa0 tx=0x7f214401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af775812ca93efab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21440b2070 con 0x7f213c0039c0 2024-04-27T07:51:55.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21440a5cc0 con 0x7f213c0039c0 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f21527fc640 1 -- 172.21.15.22:0/1903656335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144009bf0 con 0x7f213c0039c0 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/1903656335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f213c0039c0 msgr2=0x7f213c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 --2- 172.21.15.22:0/1903656335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f213c0039c0 0x7f213c003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fa0 tx=0x7f214401cc10 comp rx=0 tx=0).stop 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 -- 172.21.15.22:0/1903656335 reap_dead start 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/1903656335 shutdown_connections 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/1903656335 >> 172.21.15.22:0/1903656335 conn(0x7f2154067d10 msgr2=0x7f2154068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/1903656335 shutdown_connections 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/1903656335 wait complete. 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 Processor -- start 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- start start 2024-04-27T07:51:55.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2154152340 0x7f21540c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540cd450 0x7f21540cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21540ba2a0 con 0x7f21540b46e0 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21540ba120 con 0x7f2154152340 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f21540d4f80 con 0x7f21540cd450 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37756/0 (socket says 172.21.15.22:37756) 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 -- 172.21.15.22:0/3817832078 learned_addr learned my addr 172.21.15.22:0/3817832078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f21537fe640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2154152340 0x7f21540c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2152ffd640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540cd450 0x7f21540cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 -- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540cd450 msgr2=0x7f21540cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540cd450 0x7f21540cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 -- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2154152340 msgr2=0x7f21540c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2154152340 0x7f21540c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 -- 172.21.15.22:0/3817832078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21540c8de0 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2153fff640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f21540d0b70 tx=0x7f214c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c019020 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c003620 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c003a50 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21540c9080 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.314+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21540c9640 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2154068c30 con 0x7f21540b46e0 2024-04-27T07:51:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f214c01c020 con 0x7f21540b46e0 2024-04-27T07:51:55.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f213bfff640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 0x7f214006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f214c086bb0 con 0x7f21540b46e0 2024-04-27T07:51:55.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f21537fe640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 0x7f214006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.318+0000 7f21537fe640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 0x7f214006a870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f21440af000 tx=0x7f2144004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.322+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f214c023030 con 0x7f21540b46e0 2024-04-27T07:51:55.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.486+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21540d0030 con 0x7f21540b46e0 2024-04-27T07:51:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.486+0000 7f213bfff640 1 -- 172.21.15.22:0/3817832078 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16658 (secure 0 0 0) 0x7f214c057160 con 0x7f21540b46e0 2024-04-27T07:51:55.517 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:55.517 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:54.480991+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 msgr2=0x7f214006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 0x7f214006a870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f21440af000 tx=0x7f2144004ac0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 msgr2=0x7f21540b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f21540d0b70 tx=0x7f214c011920 comp rx=0 tx=0).stop 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 shutdown_connections 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f21400683b0 0x7f214006a870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21540b46e0 0x7f21540b4b60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21540cd450 0x7f21540cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 --2- 172.21.15.22:0/3817832078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2154152340 0x7f21540c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 >> 172.21.15.22:0/3817832078 conn(0x7f2154067d10 msgr2=0x7f21540b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 shutdown_connections 2024-04-27T07:51:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.490+0000 7f2159bbc640 1 -- 172.21.15.22:0/3817832078 wait complete. 2024-04-27T07:51:55.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:55.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.630+0000 7f31929ab640 1 Processor -- start 2024-04-27T07:51:55.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.630+0000 7f31929ab640 1 -- start start 2024-04-27T07:51:55.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f31929ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f31929ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f31929ab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c15cba0 con 0x7f318c0b1db0 2024-04-27T07:51:55.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f31929ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f318c0cd1e0 con 0x7f318c1609b0 2024-04-27T07:51:55.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f31929ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0cd4c0 con 0x7f318c15c710 2024-04-27T07:51:55.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f318b7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f3190f48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f318b7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:47424/0 (socket says 172.21.15.22:47424) 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f318b7fe640 1 -- 172.21.15.22:0/2928323863 learned_addr learned my addr 172.21.15.22:0/2928323863 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f318bfff640 1 --1- 172.21.15.22:0/2928323863 >> v1:172.21.15.183:6789/0 conn(0x7f318c0b1db0 0x7f318c1602a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:53458/0 (socket says 172.21.15.22:53458) 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f3190f48640 1 -- 172.21.15.22:0/2928323863 >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 msgr2=0x7f318c15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f3190f48640 1 --2- 172.21.15.22:0/2928323863 >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f3190f48640 1 -- 172.21.15.22:0/2928323863 >> v1:172.21.15.183:6789/0 conn(0x7f318c0b1db0 legacy=0x7f318c1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f3190f48640 1 -- 172.21.15.22:0/2928323863 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c0cd7a0 con 0x7f318c15c710 2024-04-27T07:51:55.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.634+0000 7f318b7fe640 1 --2- 172.21.15.22:0/2928323863 >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:51:55.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f3190f48640 1 --2- 172.21.15.22:0/2928323863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0b3240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3184009d90 tx=0x7f318401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8205916e6f3e36b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f318affd640 1 -- 172.21.15.22:0/2928323863 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840b2070 con 0x7f318c15c710 2024-04-27T07:51:55.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f318affd640 1 -- 172.21.15.22:0/2928323863 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31840a5990 con 0x7f318c15c710 2024-04-27T07:51:55.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f318affd640 1 -- 172.21.15.22:0/2928323863 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840a5d60 con 0x7f318c15c710 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- 172.21.15.22:0/2928323863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 msgr2=0x7f318c0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- 172.21.15.22:0/2928323863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0b3240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3184009d90 tx=0x7f318401c8b0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- 172.21.15.22:0/2928323863 shutdown_connections 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- 172.21.15.22:0/2928323863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0b3240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- 172.21.15.22:0/2928323863 >> v2:172.21.15.183:3300/0 conn(0x7f318c1609b0 0x7f318c15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- 172.21.15.22:0/2928323863 >> 172.21.15.22:0/2928323863 conn(0x7f318c0d2580 msgr2=0x7f318c0d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- 172.21.15.22:0/2928323863 shutdown_connections 2024-04-27T07:51:55.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- 172.21.15.22:0/2928323863 wait complete. 2024-04-27T07:51:55.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 Processor -- start 2024-04-27T07:51:55.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- start start 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 0x7f318c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0ce930 con 0x7f318c1609b0 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0ce7b0 con 0x7f318c0b1db0 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.638+0000 7f31929ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0ceab0 con 0x7f318c15c710 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37760/0 (socket says 172.21.15.22:37760) 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 -- 172.21.15.22:0/891301993 learned_addr learned my addr 172.21.15.22:0/891301993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f318b7fe640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f318bfff640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 0x7f318c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 -- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 msgr2=0x7f318c0c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 -- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 msgr2=0x7f318c14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 0x7f318c14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 -- 172.21.15.22:0/891301993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c14ebb0 con 0x7f318c1609b0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f318bfff640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 0x7f318c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3190f48640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f31840b0040 tx=0x7f3184019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840b2050 con 0x7f318c1609b0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31840be070 con 0x7f318c1609b0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840ada50 con 0x7f318c1609b0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c14ee90 con 0x7f318c1609b0 2024-04-27T07:51:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c1560c0 con 0x7f318c1609b0 2024-04-27T07:51:55.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c069340 con 0x7f318c1609b0 2024-04-27T07:51:55.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f31840adbf0 con 0x7f318c1609b0 2024-04-27T07:51:55.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 0x7f316c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3184121b40 con 0x7f318c1609b0 2024-04-27T07:51:55.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.642+0000 7f318bfff640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 0x7f316c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.646+0000 7f318bfff640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 0x7f316c06a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3174000f70 tx=0x7f3174007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.646+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3184016560 con 0x7f318c1609b0 2024-04-27T07:51:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.806+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f318c0d0790 con 0x7f318c1609b0 2024-04-27T07:51:55.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.810+0000 7f3188ff9640 1 -- 172.21.15.22:0/891301993 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16658 (secure 0 0 0) 0x7f31840f2220 con 0x7f318c1609b0 2024-04-27T07:51:55.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:55.818 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:54.480991+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 msgr2=0x7f316c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 0x7f316c06a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3174000f70 tx=0x7f3174007d70 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 msgr2=0x7f318c0c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f31840b0040 tx=0x7f3184019650 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 shutdown_connections 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f316c068380 0x7f316c06a840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f318c1609b0 0x7f318c0c5530 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f318c15c710 0x7f318c0c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 --2- 172.21.15.22:0/891301993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f318c0b1db0 0x7f318c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 >> 172.21.15.22:0/891301993 conn(0x7f318c0d2580 msgr2=0x7f318c0b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 shutdown_connections 2024-04-27T07:51:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.814+0000 7f31929ab640 1 -- 172.21.15.22:0/891301993 wait complete. 2024-04-27T07:51:55.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-27T07:51:55.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 Processor -- start 2024-04-27T07:51:55.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 -- start start 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f728015ecd0 0x7f7280072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 0x7f72800b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72800b5390 con 0x7f72800b3a10 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f72800b5670 con 0x7f728015ecd0 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f7284b6b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f72800ce340 con 0x7f72800b4ee0 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f728015ecd0 0x7f7280072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727ed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f72800b3a10 0x7f72800cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50518/0 (socket says 172.21.15.22:50518) 2024-04-27T07:51:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727ed9d640 1 -- 172.21.15.22:0/417675931 learned_addr learned my addr 172.21.15.22:0/417675931 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 --2- 172.21.15.22:0/417675931 >> v2:172.21.15.22:3300/0 conn(0x7f728015ecd0 0x7f7280072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37774/0 (socket says 172.21.15.22:37774) 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 -- 172.21.15.22:0/417675931 >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 msgr2=0x7f72800b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727dd9b640 1 --2- 172.21.15.22:0/417675931 >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 0x7f72800b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727d59a640 1 -- 172.21.15.22:0/417675931 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957596025 0 0) 0x7f72800b5390 con 0x7f72800b3a10 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 --2- 172.21.15.22:0/417675931 >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 0x7f72800b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 -- 172.21.15.22:0/417675931 >> v1:172.21.15.22:6789/0 conn(0x7f72800b3a10 legacy=0x7f72800cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727e59c640 1 -- 172.21.15.22:0/417675931 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72800ce620 con 0x7f728015ecd0 2024-04-27T07:51:55.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.942+0000 7f727dd9b640 1 --2- 172.21.15.22:0/417675931 >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 0x7f72800b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- 172.21.15.22:0/417675931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f728015ecd0 0x7f7280072980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7280073820 tx=0x7f726801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f399f43c329a067 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727d59a640 1 -- 172.21.15.22:0/417675931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680a7070 con 0x7f728015ecd0 2024-04-27T07:51:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727d59a640 1 -- 172.21.15.22:0/417675931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72680b0070 con 0x7f728015ecd0 2024-04-27T07:51:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727d59a640 1 -- 172.21.15.22:0/417675931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680ab890 con 0x7f728015ecd0 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/417675931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f728015ecd0 msgr2=0x7f7280072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- 172.21.15.22:0/417675931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f728015ecd0 0x7f7280072980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7280073820 tx=0x7f726801c450 comp rx=0 tx=0).stop 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/417675931 shutdown_connections 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- 172.21.15.22:0/417675931 >> v2:172.21.15.183:3300/0 conn(0x7f72800b4ee0 0x7f72800b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- 172.21.15.22:0/417675931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f728015ecd0 0x7f7280072980 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/417675931 >> 172.21.15.22:0/417675931 conn(0x7f72800b21d0 msgr2=0x7f72800b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/417675931 shutdown_connections 2024-04-27T07:51:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/417675931 wait complete. 2024-04-27T07:51:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 Processor -- start 2024-04-27T07:51:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- start start 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72800b4ee0 0x7f72800cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f728015ecd0 0x7f72801444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7280168540 con 0x7f72800b3a10 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72801686c0 con 0x7f72800b4ee0 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7280168840 con 0x7f728015ecd0 2024-04-27T07:51:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37782/0 (socket says 172.21.15.22:37782) 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 -- 172.21.15.22:0/3946840789 learned_addr learned my addr 172.21.15.22:0/3946840789 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 -- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f728015ecd0 msgr2=0x7f72801444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f728015ecd0 0x7f72801444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 -- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72800b4ee0 msgr2=0x7f72800cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72800b4ee0 0x7f72800cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 -- 172.21.15.22:0/3946840789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7280156d30 con 0x7f72800b3a10 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f727e59c640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f72680ad040 tx=0x7f726801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680bf610 con 0x7f72800b3a10 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72680bf7d0 con 0x7f72800b3a10 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.946+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7280157010 con 0x7f72800b3a10 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680bfbc0 con 0x7f72800b3a10 2024-04-27T07:51:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801575d0 con 0x7f72800b3a10 2024-04-27T07:51:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728004ff10 con 0x7f72800b3a10 2024-04-27T07:51:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f72680bd020 con 0x7f72800b3a10 2024-04-27T07:51:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7276ffd640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 0x7f726006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7268121280 con 0x7f72800b3a10 2024-04-27T07:51:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f727dd9b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 0x7f726006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.950+0000 7f727dd9b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 0x7f726006a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7280159800 tx=0x7f7270008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:55.954+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7268016070 con 0x7f72800b3a10 2024-04-27T07:51:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:56.146+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f72800cd8f0 con 0x7f72800b3a10 2024-04-27T07:51:57.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:57.145 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7276ffd640 1 -- 172.21.15.22:0/3946840789 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f72680f17a0 con 0x7f72800b3a10 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 msgr2=0x7f726006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 0x7f726006a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7280159800 tx=0x7f7270008040 comp rx=0 tx=0).stop 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 msgr2=0x7f7280156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f72680ad040 tx=0x7f726801ac90 comp rx=0 tx=0).stop 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 shutdown_connections 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7260068380 0x7f726006a840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f728015ecd0 0x7f72801444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72800b4ee0 0x7f72800cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 --2- 172.21.15.22:0/3946840789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800b3a10 0x7f7280156750 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 >> 172.21.15.22:0/3946840789 conn(0x7f72800b21d0 msgr2=0x7f72800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 shutdown_connections 2024-04-27T07:51:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.150+0000 7f7284b6b640 1 -- 172.21.15.22:0/3946840789 wait complete. 2024-04-27T07:51:57.180 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-27T07:51:57.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-27T07:51:57.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 Processor -- start 2024-04-27T07:51:57.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 -- start start 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0cb00ccf90 0x7f0cb006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb006c110 con 0x7f0cb00cd6e0 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb0070260 con 0x7f0cb00cbb10 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0cb6860640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb006c3b0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0cb00ccf90 0x7f0cb006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caf7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0cb00cd6e0 0x7f0cb006fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50520/0 (socket says 172.21.15.22:50520) 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0cb00ccf90 0x7f0cb006bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37794/0 (socket says 172.21.15.22:37794) 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1736398553 learned_addr learned my addr 172.21.15.22:0/1736398553 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caffff640 1 -- 172.21.15.22:0/1736398553 >> v1:172.21.15.183:6789/0 conn(0x7f0cb00cbb10 legacy=0x7f0cb00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caffff640 1 -- 172.21.15.22:0/1736398553 >> v1:172.21.15.22:6789/0 conn(0x7f0cb00cd6e0 legacy=0x7f0cb006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.350+0000 7f0caffff640 1 -- 172.21.15.22:0/1736398553 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb006c690 con 0x7f0cb00ccf90 2024-04-27T07:51:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0caffff640 1 --2- 172.21.15.22:0/1736398553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb006bbd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009d90 tx=0x7f0ca801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351d21e442c47355 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0caeffd640 1 -- 172.21.15.22:0/1736398553 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80b2070 con 0x7f0cb00ccf90 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0caeffd640 1 -- 172.21.15.22:0/1736398553 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca80a57f0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0caeffd640 1 -- 172.21.15.22:0/1736398553 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80a5bc0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- 172.21.15.22:0/1736398553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 msgr2=0x7f0cb006bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 --2- 172.21.15.22:0/1736398553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb006bbd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009d90 tx=0x7f0ca801c930 comp rx=0 tx=0).stop 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- 172.21.15.22:0/1736398553 shutdown_connections 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 --2- 172.21.15.22:0/1736398553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb006bbd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- 172.21.15.22:0/1736398553 >> 172.21.15.22:0/1736398553 conn(0x7f0cb009aa30 msgr2=0x7f0cb00b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- 172.21.15.22:0/1736398553 shutdown_connections 2024-04-27T07:51:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- 172.21.15.22:0/1736398553 wait complete. 2024-04-27T07:51:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 Processor -- start 2024-04-27T07:51:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.354+0000 7f0cb6860640 1 -- start start 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 0x7f0cb00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 0x7f0cb01570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb016c790 con 0x7f0cb00ccf90 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb016c910 con 0x7f0cb00cd6e0 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb016ca90 con 0x7f0cb00cbb10 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37810/0 (socket says 172.21.15.22:37810) 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 -- 172.21.15.22:0/3360645580 learned_addr learned my addr 172.21.15.22:0/3360645580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caffff640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 0x7f0cb00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb4dfd640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 0x7f0cb01570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 -- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 msgr2=0x7f0cb00c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 0x7f0cb00c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 -- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 msgr2=0x7f0cb01570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 0x7f0cb01570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 -- 172.21.15.22:0/3360645580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb0157620 con 0x7f0cb00ccf90 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caffff640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 0x7f0cb00c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb4dfd640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 0x7f0cb01570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caf7fe640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c420 tx=0x7f0ca400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4017070 con 0x7f0cb00ccf90 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca4005cf0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb00c9610 con 0x7f0cb00ccf90 2024-04-27T07:51:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40072f0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb00c9bd0 con 0x7f0cb00ccf90 2024-04-27T07:51:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ca401a070 con 0x7f0cb00ccf90 2024-04-27T07:51:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb004ff10 con 0x7f0cb00ccf90 2024-04-27T07:51:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 0x7f0c9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0ca4085d90 con 0x7f0cb00ccf90 2024-04-27T07:51:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.358+0000 7f0caffff640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 0x7f0c9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.366+0000 7f0caffff640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 0x7f0c9c06a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80af000 tx=0x7f0ca80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.366+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0cb0157620 con 0x7f0cb00ccf90 2024-04-27T07:51:57.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:57.562+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0cb015b290 con 0x7f0cb00ccf90 2024-04-27T07:51:58.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.166+0000 7f0cacff9640 1 -- 172.21.15.22:0/3360645580 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f0ca4056340 con 0x7f0cb00ccf90 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 msgr2=0x7f0c9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 0x7f0c9c06a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80af000 tx=0x7f0ca80a7070 comp rx=0 tx=0).stop 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 msgr2=0x7f0cb0156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c420 tx=0x7f0ca400c8e0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 shutdown_connections 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0c9c068380 0x7f0c9c06a840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cb00cd6e0 0x7f0cb01570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cb00ccf90 0x7f0cb0156ba0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 --2- 172.21.15.22:0/3360645580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cb00cbb10 0x7f0cb00c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 >> 172.21.15.22:0/3360645580 conn(0x7f0cb009aa30 msgr2=0x7f0cb00d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 shutdown_connections 2024-04-27T07:51:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.170+0000 7f0cb6860640 1 -- 172.21.15.22:0/3360645580 wait complete. 2024-04-27T07:51:58.194 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-27T07:51:58.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:51:58.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 Processor -- start 2024-04-27T07:51:58.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 -- start start 2024-04-27T07:51:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7eff50072740 0x7eff50072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5006e050 con 0x7eff50145a00 2024-04-27T07:51:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff500ceae0 con 0x7eff501454b0 2024-04-27T07:51:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff57690640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7eff500cedc0 con 0x7eff50072740 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7eff50072740 0x7eff50072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7eff50072740 0x7eff50072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37820/0 (socket says 172.21.15.22:37820) 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff55c2d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7eff50145a00 0x7eff5015cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50522/0 (socket says 172.21.15.22:50522) 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 -- 172.21.15.22:0/689979340 learned_addr learned my addr 172.21.15.22:0/689979340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 -- 172.21.15.22:0/689979340 >> v1:172.21.15.183:6789/0 conn(0x7eff501454b0 legacy=0x7eff50072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 -- 172.21.15.22:0/689979340 >> v1:172.21.15.22:6789/0 conn(0x7eff50145a00 legacy=0x7eff5015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff54c2b640 1 -- 172.21.15.22:0/689979340 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5006e310 con 0x7eff50072740 2024-04-27T07:51:58.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.318+0000 7eff4ffff640 1 -- 172.21.15.22:0/689979340 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898824704 0 0) 0x7eff5006e050 con 0x7eff50145a00 2024-04-27T07:51:58.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 --2- 172.21.15.22:0/689979340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff50072740 0x7eff50072fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff44009870 tx=0x7eff4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798b4eb06b8ad570 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff4ffff640 1 -- 172.21.15.22:0/689979340 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440b0070 con 0x7eff50072740 2024-04-27T07:51:58.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff4ffff640 1 -- 172.21.15.22:0/689979340 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff440ab600 con 0x7eff50072740 2024-04-27T07:51:58.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff4ffff640 1 -- 172.21.15.22:0/689979340 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440ab9d0 con 0x7eff50072740 2024-04-27T07:51:58.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- 172.21.15.22:0/689979340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff50072740 msgr2=0x7eff50072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 --2- 172.21.15.22:0/689979340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff50072740 0x7eff50072fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff44009870 tx=0x7eff4401c520 comp rx=0 tx=0).stop 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- 172.21.15.22:0/689979340 shutdown_connections 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 --2- 172.21.15.22:0/689979340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff50072740 0x7eff50072fa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- 172.21.15.22:0/689979340 >> 172.21.15.22:0/689979340 conn(0x7eff50067f10 msgr2=0x7eff5006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- 172.21.15.22:0/689979340 shutdown_connections 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- 172.21.15.22:0/689979340 wait complete. 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 Processor -- start 2024-04-27T07:51:58.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- start start 2024-04-27T07:51:58.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 0x7eff501386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 0x7eff500b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff5015d9a0 con 0x7eff501454b0 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff5015d820 con 0x7eff50072740 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff57690640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff5016ac80 con 0x7eff50145a00 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37830/0 (socket says 172.21.15.22:37830) 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 -- 172.21.15.22:0/1670178250 learned_addr learned my addr 172.21.15.22:0/1670178250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff55c2d640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 0x7eff500b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff5542c640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 0x7eff501386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 -- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 msgr2=0x7eff500b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 0x7eff500b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 -- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 msgr2=0x7eff501386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 0x7eff501386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff54c2b640 1 -- 172.21.15.22:0/1670178250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5006c9f0 con 0x7eff501454b0 2024-04-27T07:51:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff55c2d640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 0x7eff500b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.322+0000 7eff5542c640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 0x7eff501386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff54c2b640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff4401abd0 tx=0x7eff4401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440b0040 con 0x7eff501454b0 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff440bc070 con 0x7eff501454b0 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff50156220 con 0x7eff501454b0 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440b6860 con 0x7eff501454b0 2024-04-27T07:51:58.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501567e0 con 0x7eff501454b0 2024-04-27T07:51:58.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff50068c30 con 0x7eff501454b0 2024-04-27T07:51:58.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff440b6a00 con 0x7eff501454b0 2024-04-27T07:51:58.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 0x7eff3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7eff44120e10 con 0x7eff501454b0 2024-04-27T07:51:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff5542c640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 0x7eff3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.326+0000 7eff5542c640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 0x7eff3c06a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.330+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff44016110 con 0x7eff501454b0 2024-04-27T07:51:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.530+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff500afde0 con 0x7eff501454b0 2024-04-27T07:51:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff4dffb640 1 -- 172.21.15.22:0/1670178250 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7eff440f13d0 con 0x7eff501454b0 2024-04-27T07:51:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:51:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:51:58.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 msgr2=0x7eff3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 0x7eff3c06a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff48009040 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 msgr2=0x7eff5013b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff4401abd0 tx=0x7eff4401ad60 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 shutdown_connections 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7eff3c068380 0x7eff3c06a840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff50145a00 0x7eff500b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff501454b0 0x7eff5013b940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 --2- 172.21.15.22:0/1670178250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff50072740 0x7eff501386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 >> 172.21.15.22:0/1670178250 conn(0x7eff50067f10 msgr2=0x7eff5013fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 shutdown_connections 2024-04-27T07:51:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.534+0000 7eff57690640 1 -- 172.21.15.22:0/1670178250 wait complete. 2024-04-27T07:51:58.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:51:58.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.674+0000 7fc2500e0640 1 Processor -- start 2024-04-27T07:51:58.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 -- start start 2024-04-27T07:51:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc2480cea10 0x7fc2480728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc24806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc248073210 con 0x7fc248151cf0 2024-04-27T07:51:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc2481520d0 con 0x7fc2480cea10 2024-04-27T07:51:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc2500e0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc2480b3590 con 0x7fc248072e00 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24de7c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc248151cf0 0x7fc2480b5200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50532/0 (socket says 172.21.15.22:50532) 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24de7c640 1 -- 172.21.15.22:0/87109697 learned_addr learned my addr 172.21.15.22:0/87109697 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24e67d640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc24806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 --2- 172.21.15.22:0/87109697 >> v2:172.21.15.183:3300/0 conn(0x7fc2480cea10 0x7fc2480728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101182629 0 0) 0x7fc244001120 con 0x7fc248151cf0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc234003530 con 0x7fc248151cf0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4171065574 0 0) 0x7fc234003530 con 0x7fc248151cf0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 msgr2=0x7fc24806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc24806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 >> v2:172.21.15.183:3300/0 conn(0x7fc2480cea10 msgr2=0x7fc2480728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 --2- 172.21.15.22:0/87109697 >> v2:172.21.15.183:3300/0 conn(0x7fc2480cea10 0x7fc2480728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2480b3870 con 0x7fc248151cf0 2024-04-27T07:51:58.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fc244001850 con 0x7fc248151cf0 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 >> v1:172.21.15.22:6789/0 conn(0x7fc248151cf0 legacy=0x7fc2480b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc2340039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc234003f40 0x7fc234004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc234004d50 0x7fc2340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481520d0 con 0x7fc234003f40 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2480b3590 con 0x7fc234004d50 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc23400ba20 con 0x7fc248072e00 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc234003f40 0x7fc234004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24de7c640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc2340039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24e67d640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc234004d50 0x7fc2340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 -- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 msgr2=0x7fc2340039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc248072e00 0x7fc2340039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 -- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc234004d50 msgr2=0x7fc2340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc234004d50 0x7fc2340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.678+0000 7fc24d67b640 1 -- 172.21.15.22:0/87109697 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc234003530 con 0x7fc234003f40 2024-04-27T07:51:58.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc24d67b640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc234003f40 0x7fc234004810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00aab0 tx=0x7fc23c01df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cca8899f67caa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b3070 con 0x7fc234003f40 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0ae440 con 0x7fc234003f40 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc24ce7a640 1 -- 172.21.15.22:0/87109697 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00b040 con 0x7fc234003f40 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- 172.21.15.22:0/87109697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc234003f40 msgr2=0x7fc234004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 --2- 172.21.15.22:0/87109697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc234003f40 0x7fc234004810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00aab0 tx=0x7fc23c01df50 comp rx=0 tx=0).stop 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc24e67d640 1 -- 172.21.15.22:0/87109697 reap_dead start 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- 172.21.15.22:0/87109697 shutdown_connections 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- 172.21.15.22:0/87109697 >> 172.21.15.22:0/87109697 conn(0x7fc248067f10 msgr2=0x7fc24806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- 172.21.15.22:0/87109697 shutdown_connections 2024-04-27T07:51:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- 172.21.15.22:0/87109697 wait complete. 2024-04-27T07:51:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 Processor -- start 2024-04-27T07:51:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.682+0000 7fc2500e0640 1 -- start start 2024-04-27T07:51:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 0x7fc24814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc24813d210 0x7fc24814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc24806f910 con 0x7fc24813d7d0 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2480b5910 con 0x7fc24813d210 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc24806e150 con 0x7fc234003f40 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37850/0 (socket says 172.21.15.22:37850) 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 -- 172.21.15.22:0/4058679912 learned_addr learned my addr 172.21.15.22:0/4058679912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24de7c640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 0x7fc24814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24e67d640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc24813d210 0x7fc24814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 -- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 msgr2=0x7fc24814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 0x7fc24814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 -- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc24813d210 msgr2=0x7fc24814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc24813d210 0x7fc24814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 -- 172.21.15.22:0/4058679912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2480b2200 con 0x7fc24813d7d0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24de7c640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 0x7fc24814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24d67b640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01dd80 tx=0x7fc23c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b3040 con 0x7fc24813d7d0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0b9630 con 0x7fc24813d7d0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b9a20 con 0x7fc24813d7d0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2480b24e0 con 0x7fc24813d7d0 2024-04-27T07:51:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24814a170 con 0x7fc24813d7d0 2024-04-27T07:51:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc248068c30 con 0x7fc24813d7d0 2024-04-27T07:51:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc23c0c0060 con 0x7fc24813d7d0 2024-04-27T07:51:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 0x7fc22c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc23c120f90 con 0x7fc24813d7d0 2024-04-27T07:51:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.686+0000 7fc24de7c640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 0x7fc22c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.690+0000 7fc24de7c640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 0x7fc22c06a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc244001300 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.690+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc23c017cd0 con 0x7fc24813d7d0 2024-04-27T07:51:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.854+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2480694b0 con 0x7fc24813d7d0 2024-04-27T07:51:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc23a7fc640 1 -- 172.21.15.22:0/4058679912 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7fc23c0f15e0 con 0x7fc24813d7d0 2024-04-27T07:51:58.860 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:58.860 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:56.149113+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:51:58.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 msgr2=0x7fc22c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 0x7fc22c06a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc244001300 tx=0x7fc244009040 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 msgr2=0x7fc2480b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01dd80 tx=0x7fc23c01acf0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 shutdown_connections 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc22c068380 0x7fc22c06a840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc24813d210 0x7fc24814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24813d7d0 0x7fc2480b0870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 --2- 172.21.15.22:0/4058679912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc234003f40 0x7fc24814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 >> 172.21.15.22:0/4058679912 conn(0x7fc248067f10 msgr2=0x7fc24814cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 shutdown_connections 2024-04-27T07:51:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:58.858+0000 7fc2500e0640 1 -- 172.21.15.22:0/4058679912 wait complete. 2024-04-27T07:51:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-27T07:51:58.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:51:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 Processor -- start 2024-04-27T07:51:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 -- start start 2024-04-27T07:51:59.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 0x7fdcd015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdcd015c710 0x7fdcd00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd015cba0 con 0x7fdcd00b1db0 2024-04-27T07:51:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd00cd1e0 con 0x7fdcd015c710 2024-04-27T07:51:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd664b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd00cd4c0 con 0x7fdcd01609b0 2024-04-27T07:51:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdcd00b1db0 0x7fdcd01602a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50548/0 (socket says 172.21.15.22:50548) 2024-04-27T07:51:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccffff640 1 -- 172.21.15.22:0/4266597379 learned_addr learned my addr 172.21.15.22:0/4266597379 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/4266597379 >> v2:172.21.15.183:3300/0 conn(0x7fdcd015c710 0x7fdcd00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcc7fff640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 0x7fdcd015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332263638 0 0) 0x7fdcc0001120 con 0x7fdcd00b1db0 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcbc003530 con 0x7fdcd00b1db0 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2980985122 0 0) 0x7fdcbc003530 con 0x7fdcd00b1db0 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 msgr2=0x7fdcd015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 0x7fdcd015c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 >> v2:172.21.15.183:3300/0 conn(0x7fdcd015c710 msgr2=0x7fdcd00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 --2- 172.21.15.22:0/4266597379 >> v2:172.21.15.183:3300/0 conn(0x7fdcd015c710 0x7fdcd00b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd00cd7a0 con 0x7fdcd00b1db0 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fdcc0001850 con 0x7fdcd00b1db0 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 >> v1:172.21.15.22:6789/0 conn(0x7fdcd00b1db0 legacy=0x7fdcd01602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 0x7fdcbc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc003f40 0x7fdcbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc004d50 0x7fdcbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd00cd1e0 con 0x7fdcbc004d50 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd00cd4c0 con 0x7fdcbc003f40 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc00ba20 con 0x7fdcd01609b0 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc004d50 0x7fdcbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcc7fff640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc003f40 0x7fdcbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 -- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 msgr2=0x7fdcbc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd01609b0 0x7fdcbc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 -- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc003f40 msgr2=0x7fdcbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc003f40 0x7fdcbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.002+0000 7fdcd4be8640 1 -- 172.21.15.22:0/4266597379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc003530 con 0x7fdcbc004d50 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc004d50 0x7fdcbc0051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800e170 tx=0x7fdcc801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeefe60ed415f490 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80b2070 con 0x7fdcbc004d50 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc80a5bf0 con 0x7fdcbc004d50 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdccf7fe640 1 -- 172.21.15.22:0/4266597379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8009c10 con 0x7fdcbc004d50 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- 172.21.15.22:0/4266597379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc004d50 msgr2=0x7fdcbc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 --2- 172.21.15.22:0/4266597379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc004d50 0x7fdcbc0051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800e170 tx=0x7fdcc801cbe0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd4be8640 1 -- 172.21.15.22:0/4266597379 reap_dead start 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- 172.21.15.22:0/4266597379 shutdown_connections 2024-04-27T07:51:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- 172.21.15.22:0/4266597379 >> 172.21.15.22:0/4266597379 conn(0x7fdcd00d2580 msgr2=0x7fdcd00d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- 172.21.15.22:0/4266597379 shutdown_connections 2024-04-27T07:51:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- 172.21.15.22:0/4266597379 wait complete. 2024-04-27T07:51:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 Processor -- start 2024-04-27T07:51:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- start start 2024-04-27T07:51:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc004d50 0x7fdcd00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd00d0130 0x7fdcd00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd015c1d0 con 0x7fdcd0149800 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd00cce90 con 0x7fdcbc004d50 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcd664b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd015c350 con 0x7fdcd00d0130 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.006+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37866/0 (socket says 172.21.15.22:37866) 2024-04-27T07:51:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 -- 172.21.15.22:0/2557517125 learned_addr learned my addr 172.21.15.22:0/2557517125 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcd4be8640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd00d0130 0x7fdcd00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 -- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd00d0130 msgr2=0x7fdcd00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd00d0130 0x7fdcd00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 -- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc004d50 msgr2=0x7fdcd00c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc004d50 0x7fdcd00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 -- 172.21.15.22:0/2557517125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd00cbae0 con 0x7fdcd0149800 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcc7fff640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011680 tx=0x7fdcb8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800f9c0 con 0x7fdcd0149800 2024-04-27T07:51:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb800fb60 con 0x7fdcd0149800 2024-04-27T07:51:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8015460 con 0x7fdcd0149800 2024-04-27T07:51:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd00cbdc0 con 0x7fdcd0149800 2024-04-27T07:51:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd00cc380 con 0x7fdcd0149800 2024-04-27T07:51:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd0069340 con 0x7fdcd0149800 2024-04-27T07:51:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcb8015600 con 0x7fdcd0149800 2024-04-27T07:51:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 0x7fdcb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb80872a0 con 0x7fdcd0149800 2024-04-27T07:51:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccffff640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 0x7fdcb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.010+0000 7fdccffff640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 0x7fdcb006a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0000f70 tx=0x7fdcc00011c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.014+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcb8019030 con 0x7fdcd0149800 2024-04-27T07:51:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.166+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcd0071ea0 con 0x7fdcb0068380 2024-04-27T07:51:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.186+0000 7fdccd7fa640 1 -- 172.21.15.22:0/2557517125 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7fdcd0071ea0 con 0x7fdcb0068380 2024-04-27T07:51:59.194 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:51:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 msgr2=0x7fdcb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 0x7fdcb006a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0000f70 tx=0x7fdcc00011c0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 msgr2=0x7fdcd00cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011680 tx=0x7fdcb8011b40 comp rx=0 tx=0).stop 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 shutdown_connections 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdcb0068380 0x7fdcb006a840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcd00d0130 0x7fdcd00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd0149800 0x7fdcd00cb1b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 --2- 172.21.15.22:0/2557517125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcbc004d50 0x7fdcd00c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 >> 172.21.15.22:0/2557517125 conn(0x7fdcd00d2580 msgr2=0x7fdcd015ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 shutdown_connections 2024-04-27T07:51:59.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.190+0000 7fdcd664b640 1 -- 172.21.15.22:0/2557517125 wait complete. 2024-04-27T07:51:59.211 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-27T07:51:58.171469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241176,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733544,"statfs":{"total":773094113280,"available":772847149056,"internally_reserved":0,"allocated":4300800,"data_stored":2384252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078600"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361085+0000","last_change":"2024-04-27T07:51:55.188412+0000","last_active":"2024-04-27T07:51:56.361085+0000","last_peered":"2024-04-27T07:51:56.361085+0000","last_clean":"2024-04-27T07:51:56.361085+0000","last_became_active":"2024-04-27T07:51:55.186996+0000","last_became_peered":"2024-04-27T07:51:55.186996+0000","last_unstale":"2024-04-27T07:51:56.361085+0000","last_undegraded":"2024-04-27T07:51:56.361085+0000","last_fullsized":"2024-04-27T07:51:56.361085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:49.764377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358126+0000","last_change":"2024-04-27T07:51:55.172585+0000","last_active":"2024-04-27T07:51:56.358126+0000","last_peered":"2024-04-27T07:51:56.358126+0000","last_clean":"2024-04-27T07:51:56.358126+0000","last_became_active":"2024-04-27T07:51:55.172067+0000","last_became_peered":"2024-04-27T07:51:55.172067+0000","last_unstale":"2024-04-27T07:51:56.358126+0000","last_undegraded":"2024-04-27T07:51:56.358126+0000","last_fullsized":"2024-04-27T07:51:56.358126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:05:35.477534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361914+0000","last_change":"2024-04-27T07:51:55.167000+0000","last_active":"2024-04-27T07:51:56.361914+0000","last_peered":"2024-04-27T07:51:56.361914+0000","last_clean":"2024-04-27T07:51:56.361914+0000","last_became_active":"2024-04-27T07:51:55.165675+0000","last_became_peered":"2024-04-27T07:51:55.165675+0000","last_unstale":"2024-04-27T07:51:56.361914+0000","last_undegraded":"2024-04-27T07:51:56.361914+0000","last_fullsized":"2024-04-27T07:51:56.361914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:29.864584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163375+0000","last_change":"2024-04-27T07:51:55.197784+0000","last_active":"2024-04-27T07:51:56.163375+0000","last_peered":"2024-04-27T07:51:56.163375+0000","last_clean":"2024-04-27T07:51:56.163375+0000","last_became_active":"2024-04-27T07:51:55.197481+0000","last_became_peered":"2024-04-27T07:51:55.197481+0000","last_unstale":"2024-04-27T07:51:56.163375+0000","last_undegraded":"2024-04-27T07:51:56.163375+0000","last_fullsized":"2024-04-27T07:51:56.163375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:31.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154687+0000","last_change":"2024-04-27T07:51:55.172087+0000","last_active":"2024-04-27T07:51:56.154687+0000","last_peered":"2024-04-27T07:51:56.154687+0000","last_clean":"2024-04-27T07:51:56.154687+0000","last_became_active":"2024-04-27T07:51:55.171808+0000","last_became_peered":"2024-04-27T07:51:55.171808+0000","last_unstale":"2024-04-27T07:51:56.154687+0000","last_undegraded":"2024-04-27T07:51:56.154687+0000","last_fullsized":"2024-04-27T07:51:56.154687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:07.865042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357667+0000","last_change":"2024-04-27T07:51:55.154885+0000","last_active":"2024-04-27T07:51:56.357667+0000","last_peered":"2024-04-27T07:51:56.357667+0000","last_clean":"2024-04-27T07:51:56.357667+0000","last_became_active":"2024-04-27T07:51:55.154369+0000","last_became_peered":"2024-04-27T07:51:55.154369+0000","last_unstale":"2024-04-27T07:51:56.357667+0000","last_undegraded":"2024-04-27T07:51:56.357667+0000","last_fullsized":"2024-04-27T07:51:56.357667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:51:36.231986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359562+0000","last_change":"2024-04-27T07:51:55.186506+0000","last_active":"2024-04-27T07:51:56.359562+0000","last_peered":"2024-04-27T07:51:56.359562+0000","last_clean":"2024-04-27T07:51:56.359562+0000","last_became_active":"2024-04-27T07:51:55.184683+0000","last_became_peered":"2024-04-27T07:51:55.184683+0000","last_unstale":"2024-04-27T07:51:56.359562+0000","last_undegraded":"2024-04-27T07:51:56.359562+0000","last_fullsized":"2024-04-27T07:51:56.359562+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:31.927909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358783+0000","last_change":"2024-04-27T07:51:55.205178+0000","last_active":"2024-04-27T07:51:56.358783+0000","last_peered":"2024-04-27T07:51:56.358783+0000","last_clean":"2024-04-27T07:51:56.358783+0000","last_became_active":"2024-04-27T07:51:55.204662+0000","last_became_peered":"2024-04-27T07:51:55.204662+0000","last_unstale":"2024-04-27T07:51:56.358783+0000","last_undegraded":"2024-04-27T07:51:56.358783+0000","last_fullsized":"2024-04-27T07:51:56.358783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:38.267349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159665+0000","last_change":"2024-04-27T07:51:55.190539+0000","last_active":"2024-04-27T07:51:56.159665+0000","last_peered":"2024-04-27T07:51:56.159665+0000","last_clean":"2024-04-27T07:51:56.159665+0000","last_became_active":"2024-04-27T07:51:55.189855+0000","last_became_peered":"2024-04-27T07:51:55.189855+0000","last_unstale":"2024-04-27T07:51:56.159665+0000","last_undegraded":"2024-04-27T07:51:56.159665+0000","last_fullsized":"2024-04-27T07:51:56.159665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:23:20.564940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163390+0000","last_change":"2024-04-27T07:51:55.167873+0000","last_active":"2024-04-27T07:51:56.163390+0000","last_peered":"2024-04-27T07:51:56.163390+0000","last_clean":"2024-04-27T07:51:56.163390+0000","last_became_active":"2024-04-27T07:51:55.167578+0000","last_became_peered":"2024-04-27T07:51:55.167578+0000","last_unstale":"2024-04-27T07:51:56.163390+0000","last_undegraded":"2024-04-27T07:51:56.163390+0000","last_fullsized":"2024-04-27T07:51:56.163390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:30.207284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361376+0000","last_change":"2024-04-27T07:51:55.185096+0000","last_active":"2024-04-27T07:51:56.361376+0000","last_peered":"2024-04-27T07:51:56.361376+0000","last_clean":"2024-04-27T07:51:56.361376+0000","last_became_active":"2024-04-27T07:51:55.164596+0000","last_became_peered":"2024-04-27T07:51:55.164596+0000","last_unstale":"2024-04-27T07:51:56.361376+0000","last_undegraded":"2024-04-27T07:51:56.361376+0000","last_fullsized":"2024-04-27T07:51:56.361376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:45.090148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361188+0000","last_change":"2024-04-27T07:51:55.175952+0000","last_active":"2024-04-27T07:51:56.361188+0000","last_peered":"2024-04-27T07:51:56.361188+0000","last_clean":"2024-04-27T07:51:56.361188+0000","last_became_active":"2024-04-27T07:51:55.174964+0000","last_became_peered":"2024-04-27T07:51:55.174964+0000","last_unstale":"2024-04-27T07:51:56.361188+0000","last_undegraded":"2024-04-27T07:51:56.361188+0000","last_fullsized":"2024-04-27T07:51:56.361188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:05:35.477534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154879+0000","last_change":"2024-04-27T07:51:55.163174+0000","last_active":"2024-04-27T07:51:56.154879+0000","last_peered":"2024-04-27T07:51:56.154879+0000","last_clean":"2024-04-27T07:51:56.154879+0000","last_became_active":"2024-04-27T07:51:55.162870+0000","last_became_peered":"2024-04-27T07:51:55.162870+0000","last_unstale":"2024-04-27T07:51:56.154879+0000","last_undegraded":"2024-04-27T07:51:56.154879+0000","last_fullsized":"2024-04-27T07:51:56.154879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:14.061315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357731+0000","last_change":"2024-04-27T07:51:55.171896+0000","last_active":"2024-04-27T07:51:56.357731+0000","last_peered":"2024-04-27T07:51:56.357731+0000","last_clean":"2024-04-27T07:51:56.357731+0000","last_became_active":"2024-04-27T07:51:55.171589+0000","last_became_peered":"2024-04-27T07:51:55.171589+0000","last_unstale":"2024-04-27T07:51:56.357731+0000","last_undegraded":"2024-04-27T07:51:56.357731+0000","last_fullsized":"2024-04-27T07:51:56.357731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:32.692411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360283+0000","last_change":"2024-04-27T07:51:55.169407+0000","last_active":"2024-04-27T07:51:56.360283+0000","last_peered":"2024-04-27T07:51:56.360283+0000","last_clean":"2024-04-27T07:51:56.360283+0000","last_became_active":"2024-04-27T07:51:55.168695+0000","last_became_peered":"2024-04-27T07:51:55.168695+0000","last_unstale":"2024-04-27T07:51:56.360283+0000","last_undegraded":"2024-04-27T07:51:56.360283+0000","last_fullsized":"2024-04-27T07:51:56.360283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:20.305883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359370+0000","last_change":"2024-04-27T07:51:55.207162+0000","last_active":"2024-04-27T07:51:56.359370+0000","last_peered":"2024-04-27T07:51:56.359370+0000","last_clean":"2024-04-27T07:51:56.359370+0000","last_became_active":"2024-04-27T07:51:55.206324+0000","last_became_peered":"2024-04-27T07:51:55.206324+0000","last_unstale":"2024-04-27T07:51:56.359370+0000","last_undegraded":"2024-04-27T07:51:56.359370+0000","last_fullsized":"2024-04-27T07:51:56.359370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:46.584110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361689+0000","last_change":"2024-04-27T07:51:55.179158+0000","last_active":"2024-04-27T07:51:56.361689+0000","last_peered":"2024-04-27T07:51:56.361689+0000","last_clean":"2024-04-27T07:51:56.361689+0000","last_became_active":"2024-04-27T07:51:55.177884+0000","last_became_peered":"2024-04-27T07:51:55.177884+0000","last_unstale":"2024-04-27T07:51:56.361689+0000","last_undegraded":"2024-04-27T07:51:56.361689+0000","last_fullsized":"2024-04-27T07:51:56.361689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:38.648029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358331+0000","last_change":"2024-04-27T07:51:55.159436+0000","last_active":"2024-04-27T07:51:56.358331+0000","last_peered":"2024-04-27T07:51:56.358331+0000","last_clean":"2024-04-27T07:51:56.358331+0000","last_became_active":"2024-04-27T07:51:55.159137+0000","last_became_peered":"2024-04-27T07:51:55.159137+0000","last_unstale":"2024-04-27T07:51:56.358331+0000","last_undegraded":"2024-04-27T07:51:56.358331+0000","last_fullsized":"2024-04-27T07:51:56.358331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:59:32.806059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357906+0000","last_change":"2024-04-27T07:51:55.165966+0000","last_active":"2024-04-27T07:51:56.357906+0000","last_peered":"2024-04-27T07:51:56.357906+0000","last_clean":"2024-04-27T07:51:56.357906+0000","last_became_active":"2024-04-27T07:51:55.162244+0000","last_became_peered":"2024-04-27T07:51:55.162244+0000","last_unstale":"2024-04-27T07:51:56.357906+0000","last_undegraded":"2024-04-27T07:51:56.357906+0000","last_fullsized":"2024-04-27T07:51:56.357906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:34.465556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154683+0000","last_change":"2024-04-27T07:51:55.172770+0000","last_active":"2024-04-27T07:51:56.154683+0000","last_peered":"2024-04-27T07:51:56.154683+0000","last_clean":"2024-04-27T07:51:56.154683+0000","last_became_active":"2024-04-27T07:51:55.172473+0000","last_became_peered":"2024-04-27T07:51:55.172473+0000","last_unstale":"2024-04-27T07:51:56.154683+0000","last_undegraded":"2024-04-27T07:51:56.154683+0000","last_fullsized":"2024-04-27T07:51:56.154683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:51.418430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158599+0000","last_change":"2024-04-27T07:51:55.191317+0000","last_active":"2024-04-27T07:51:56.158599+0000","last_peered":"2024-04-27T07:51:56.158599+0000","last_clean":"2024-04-27T07:51:56.158599+0000","last_became_active":"2024-04-27T07:51:55.190428+0000","last_became_peered":"2024-04-27T07:51:55.190428+0000","last_unstale":"2024-04-27T07:51:56.158599+0000","last_undegraded":"2024-04-27T07:51:56.158599+0000","last_fullsized":"2024-04-27T07:51:56.158599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:48.739020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158129+0000","last_change":"2024-04-27T07:51:55.192573+0000","last_active":"2024-04-27T07:51:56.158129+0000","last_peered":"2024-04-27T07:51:56.158129+0000","last_clean":"2024-04-27T07:51:56.158129+0000","last_became_active":"2024-04-27T07:51:55.191795+0000","last_became_peered":"2024-04-27T07:51:55.191795+0000","last_unstale":"2024-04-27T07:51:56.158129+0000","last_undegraded":"2024-04-27T07:51:56.158129+0000","last_fullsized":"2024-04-27T07:51:56.158129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:28.176405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155744+0000","last_change":"2024-04-27T07:51:55.174727+0000","last_active":"2024-04-27T07:51:56.155744+0000","last_peered":"2024-04-27T07:51:56.155744+0000","last_clean":"2024-04-27T07:51:56.155744+0000","last_became_active":"2024-04-27T07:51:55.174292+0000","last_became_peered":"2024-04-27T07:51:55.174292+0000","last_unstale":"2024-04-27T07:51:56.155744+0000","last_undegraded":"2024-04-27T07:51:56.155744+0000","last_fullsized":"2024-04-27T07:51:56.155744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:23:44.174069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358921+0000","last_change":"2024-04-27T07:51:55.168229+0000","last_active":"2024-04-27T07:51:56.358921+0000","last_peered":"2024-04-27T07:51:56.358921+0000","last_clean":"2024-04-27T07:51:56.358921+0000","last_became_active":"2024-04-27T07:51:55.167503+0000","last_became_peered":"2024-04-27T07:51:55.167503+0000","last_unstale":"2024-04-27T07:51:56.358921+0000","last_undegraded":"2024-04-27T07:51:56.358921+0000","last_fullsized":"2024-04-27T07:51:56.358921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:31.842640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163568+0000","last_change":"2024-04-27T07:51:55.175811+0000","last_active":"2024-04-27T07:51:56.163568+0000","last_peered":"2024-04-27T07:51:56.163568+0000","last_clean":"2024-04-27T07:51:56.163568+0000","last_became_active":"2024-04-27T07:51:55.175525+0000","last_became_peered":"2024-04-27T07:51:55.175525+0000","last_unstale":"2024-04-27T07:51:56.163568+0000","last_undegraded":"2024-04-27T07:51:56.163568+0000","last_fullsized":"2024-04-27T07:51:56.163568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:23:52.641179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358534+0000","last_change":"2024-04-27T07:51:55.176629+0000","last_active":"2024-04-27T07:51:56.358534+0000","last_peered":"2024-04-27T07:51:56.358534+0000","last_clean":"2024-04-27T07:51:56.358534+0000","last_became_active":"2024-04-27T07:51:55.176339+0000","last_became_peered":"2024-04-27T07:51:55.176339+0000","last_unstale":"2024-04-27T07:51:56.358534+0000","last_undegraded":"2024-04-27T07:51:56.358534+0000","last_fullsized":"2024-04-27T07:51:56.358534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:34.753412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163796+0000","last_change":"2024-04-27T07:51:55.197584+0000","last_active":"2024-04-27T07:51:56.163796+0000","last_peered":"2024-04-27T07:51:56.163796+0000","last_clean":"2024-04-27T07:51:56.163796+0000","last_became_active":"2024-04-27T07:51:55.197251+0000","last_became_peered":"2024-04-27T07:51:55.197251+0000","last_unstale":"2024-04-27T07:51:56.163796+0000","last_undegraded":"2024-04-27T07:51:56.163796+0000","last_fullsized":"2024-04-27T07:51:56.163796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:03:27.316042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360945+0000","last_change":"2024-04-27T07:51:55.170496+0000","last_active":"2024-04-27T07:51:56.360945+0000","last_peered":"2024-04-27T07:51:56.360945+0000","last_clean":"2024-04-27T07:51:56.360945+0000","last_became_active":"2024-04-27T07:51:55.169457+0000","last_became_peered":"2024-04-27T07:51:55.169457+0000","last_unstale":"2024-04-27T07:51:56.360945+0000","last_undegraded":"2024-04-27T07:51:56.360945+0000","last_fullsized":"2024-04-27T07:51:56.360945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:16.089427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158632+0000","last_change":"2024-04-27T07:51:55.191102+0000","last_active":"2024-04-27T07:51:56.158632+0000","last_peered":"2024-04-27T07:51:56.158632+0000","last_clean":"2024-04-27T07:51:56.158632+0000","last_became_active":"2024-04-27T07:51:55.190216+0000","last_became_peered":"2024-04-27T07:51:55.190216+0000","last_unstale":"2024-04-27T07:51:56.158632+0000","last_undegraded":"2024-04-27T07:51:56.158632+0000","last_fullsized":"2024-04-27T07:51:56.158632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:15.797290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359208+0000","last_change":"2024-04-27T07:51:55.200839+0000","last_active":"2024-04-27T07:51:56.359208+0000","last_peered":"2024-04-27T07:51:56.359208+0000","last_clean":"2024-04-27T07:51:56.359208+0000","last_became_active":"2024-04-27T07:51:55.193370+0000","last_became_peered":"2024-04-27T07:51:55.193370+0000","last_unstale":"2024-04-27T07:51:56.359208+0000","last_undegraded":"2024-04-27T07:51:56.359208+0000","last_fullsized":"2024-04-27T07:51:56.359208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:55.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360151+0000","last_change":"2024-04-27T07:51:55.169522+0000","last_active":"2024-04-27T07:51:56.360151+0000","last_peered":"2024-04-27T07:51:56.360151+0000","last_clean":"2024-04-27T07:51:56.360151+0000","last_became_active":"2024-04-27T07:51:55.166086+0000","last_became_peered":"2024-04-27T07:51:55.166086+0000","last_unstale":"2024-04-27T07:51:56.360151+0000","last_undegraded":"2024-04-27T07:51:56.360151+0000","last_fullsized":"2024-04-27T07:51:56.360151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:50:23.363287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358951+0000","last_change":"2024-04-27T07:51:55.206346+0000","last_active":"2024-04-27T07:51:56.358951+0000","last_peered":"2024-04-27T07:51:56.358951+0000","last_clean":"2024-04-27T07:51:56.358951+0000","last_became_active":"2024-04-27T07:51:55.205742+0000","last_became_peered":"2024-04-27T07:51:55.205742+0000","last_unstale":"2024-04-27T07:51:56.358951+0000","last_undegraded":"2024-04-27T07:51:56.358951+0000","last_fullsized":"2024-04-27T07:51:56.358951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:10.146181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156740+0000","last_change":"2024-04-27T07:51:55.200655+0000","last_active":"2024-04-27T07:51:56.156740+0000","last_peered":"2024-04-27T07:51:56.156740+0000","last_clean":"2024-04-27T07:51:56.156740+0000","last_became_active":"2024-04-27T07:51:55.200348+0000","last_became_peered":"2024-04-27T07:51:55.200348+0000","last_unstale":"2024-04-27T07:51:56.156740+0000","last_undegraded":"2024-04-27T07:51:56.156740+0000","last_fullsized":"2024-04-27T07:51:56.156740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:14.466571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361232+0000","last_change":"2024-04-27T07:51:54.137725+0000","last_active":"2024-04-27T07:51:56.361232+0000","last_peered":"2024-04-27T07:51:56.361232+0000","last_clean":"2024-04-27T07:51:56.361232+0000","last_became_active":"2024-04-27T07:51:54.136080+0000","last_became_peered":"2024-04-27T07:51:54.136080+0000","last_unstale":"2024-04-27T07:51:56.361232+0000","last_undegraded":"2024-04-27T07:51:56.361232+0000","last_fullsized":"2024-04-27T07:51:56.361232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:03.819220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361243+0000","last_change":"2024-04-27T07:51:55.194268+0000","last_active":"2024-04-27T07:51:56.361243+0000","last_peered":"2024-04-27T07:51:56.361243+0000","last_clean":"2024-04-27T07:51:56.361243+0000","last_became_active":"2024-04-27T07:51:55.193615+0000","last_became_peered":"2024-04-27T07:51:55.193615+0000","last_unstale":"2024-04-27T07:51:56.361243+0000","last_undegraded":"2024-04-27T07:51:56.361243+0000","last_fullsized":"2024-04-27T07:51:56.361243+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:18:09.850466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156661+0000","last_change":"2024-04-27T07:51:54.122462+0000","last_active":"2024-04-27T07:51:56.156661+0000","last_peered":"2024-04-27T07:51:56.156661+0000","last_clean":"2024-04-27T07:51:56.156661+0000","last_became_active":"2024-04-27T07:51:54.121723+0000","last_became_peered":"2024-04-27T07:51:54.121723+0000","last_unstale":"2024-04-27T07:51:56.156661+0000","last_undegraded":"2024-04-27T07:51:56.156661+0000","last_fullsized":"2024-04-27T07:51:56.156661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:44:55.866398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159007+0000","last_change":"2024-04-27T07:51:55.197429+0000","last_active":"2024-04-27T07:51:56.159007+0000","last_peered":"2024-04-27T07:51:56.159007+0000","last_clean":"2024-04-27T07:51:56.159007+0000","last_became_active":"2024-04-27T07:51:55.191819+0000","last_became_peered":"2024-04-27T07:51:55.191819+0000","last_unstale":"2024-04-27T07:51:56.159007+0000","last_undegraded":"2024-04-27T07:51:56.159007+0000","last_fullsized":"2024-04-27T07:51:56.159007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:31.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357835+0000","last_change":"2024-04-27T07:51:54.113993+0000","last_active":"2024-04-27T07:51:56.357835+0000","last_peered":"2024-04-27T07:51:56.357835+0000","last_clean":"2024-04-27T07:51:56.357835+0000","last_became_active":"2024-04-27T07:51:54.112772+0000","last_became_peered":"2024-04-27T07:51:54.112772+0000","last_unstale":"2024-04-27T07:51:56.357835+0000","last_undegraded":"2024-04-27T07:51:56.357835+0000","last_fullsized":"2024-04-27T07:51:56.357835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:14.788660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163969+0000","last_change":"2024-04-27T07:51:55.191318+0000","last_active":"2024-04-27T07:51:56.163969+0000","last_peered":"2024-04-27T07:51:56.163969+0000","last_clean":"2024-04-27T07:51:56.163969+0000","last_became_active":"2024-04-27T07:51:55.190978+0000","last_became_peered":"2024-04-27T07:51:55.190978+0000","last_unstale":"2024-04-27T07:51:56.163969+0000","last_undegraded":"2024-04-27T07:51:56.163969+0000","last_fullsized":"2024-04-27T07:51:56.163969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:40.482291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154839+0000","last_change":"2024-04-27T07:51:54.117646+0000","last_active":"2024-04-27T07:51:56.154839+0000","last_peered":"2024-04-27T07:51:56.154839+0000","last_clean":"2024-04-27T07:51:56.154839+0000","last_became_active":"2024-04-27T07:51:54.117130+0000","last_became_peered":"2024-04-27T07:51:54.117130+0000","last_unstale":"2024-04-27T07:51:56.154839+0000","last_undegraded":"2024-04-27T07:51:56.154839+0000","last_fullsized":"2024-04-27T07:51:56.154839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:31.067921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359616+0000","last_change":"2024-04-27T07:51:55.202425+0000","last_active":"2024-04-27T07:51:56.359616+0000","last_peered":"2024-04-27T07:51:56.359616+0000","last_clean":"2024-04-27T07:51:56.359616+0000","last_became_active":"2024-04-27T07:51:55.201723+0000","last_became_peered":"2024-04-27T07:51:55.201723+0000","last_unstale":"2024-04-27T07:51:56.359616+0000","last_undegraded":"2024-04-27T07:51:56.359616+0000","last_fullsized":"2024-04-27T07:51:56.359616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:10:08.361934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158919+0000","last_change":"2024-04-27T07:51:54.138877+0000","last_active":"2024-04-27T07:51:56.158919+0000","last_peered":"2024-04-27T07:51:56.158919+0000","last_clean":"2024-04-27T07:51:56.158919+0000","last_became_active":"2024-04-27T07:51:54.137251+0000","last_became_peered":"2024-04-27T07:51:54.137251+0000","last_unstale":"2024-04-27T07:51:56.158919+0000","last_undegraded":"2024-04-27T07:51:56.158919+0000","last_fullsized":"2024-04-27T07:51:56.158919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:03.861737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360914+0000","last_change":"2024-04-27T07:51:55.170525+0000","last_active":"2024-04-27T07:51:56.360914+0000","last_peered":"2024-04-27T07:51:56.360914+0000","last_clean":"2024-04-27T07:51:56.360914+0000","last_became_active":"2024-04-27T07:51:55.169768+0000","last_became_peered":"2024-04-27T07:51:55.169768+0000","last_unstale":"2024-04-27T07:51:56.360914+0000","last_undegraded":"2024-04-27T07:51:56.360914+0000","last_fullsized":"2024-04-27T07:51:56.360914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:34.465556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158393+0000","last_change":"2024-04-27T07:51:54.130222+0000","last_active":"2024-04-27T07:51:56.158393+0000","last_peered":"2024-04-27T07:51:56.158393+0000","last_clean":"2024-04-27T07:51:56.158393+0000","last_became_active":"2024-04-27T07:51:54.129483+0000","last_became_peered":"2024-04-27T07:51:54.129483+0000","last_unstale":"2024-04-27T07:51:56.158393+0000","last_undegraded":"2024-04-27T07:51:56.158393+0000","last_fullsized":"2024-04-27T07:51:56.158393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157955+0000","last_change":"2024-04-27T07:51:55.199308+0000","last_active":"2024-04-27T07:51:56.157955+0000","last_peered":"2024-04-27T07:51:56.157955+0000","last_clean":"2024-04-27T07:51:56.157955+0000","last_became_active":"2024-04-27T07:51:55.198740+0000","last_became_peered":"2024-04-27T07:51:55.198740+0000","last_unstale":"2024-04-27T07:51:56.157955+0000","last_undegraded":"2024-04-27T07:51:56.157955+0000","last_fullsized":"2024-04-27T07:51:56.157955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:57.706999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357367+0000","last_change":"2024-04-27T07:51:54.119289+0000","last_active":"2024-04-27T07:51:56.357367+0000","last_peered":"2024-04-27T07:51:56.357367+0000","last_clean":"2024-04-27T07:51:56.357367+0000","last_became_active":"2024-04-27T07:51:54.118743+0000","last_became_peered":"2024-04-27T07:51:54.118743+0000","last_unstale":"2024-04-27T07:51:56.357367+0000","last_undegraded":"2024-04-27T07:51:56.357367+0000","last_fullsized":"2024-04-27T07:51:56.357367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:56.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163213+0000","last_change":"2024-04-27T07:51:55.184764+0000","last_active":"2024-04-27T07:51:56.163213+0000","last_peered":"2024-04-27T07:51:56.163213+0000","last_clean":"2024-04-27T07:51:56.163213+0000","last_became_active":"2024-04-27T07:51:55.184530+0000","last_became_peered":"2024-04-27T07:51:55.184530+0000","last_unstale":"2024-04-27T07:51:56.163213+0000","last_undegraded":"2024-04-27T07:51:56.163213+0000","last_fullsized":"2024-04-27T07:51:56.163213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:15.797290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157766+0000","last_change":"2024-04-27T07:51:54.143742+0000","last_active":"2024-04-27T07:51:56.157766+0000","last_peered":"2024-04-27T07:51:56.157766+0000","last_clean":"2024-04-27T07:51:56.157766+0000","last_became_active":"2024-04-27T07:51:54.138744+0000","last_became_peered":"2024-04-27T07:51:54.138744+0000","last_unstale":"2024-04-27T07:51:56.157766+0000","last_undegraded":"2024-04-27T07:51:56.157766+0000","last_fullsized":"2024-04-27T07:51:56.157766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:44.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360560+0000","last_change":"2024-04-27T07:51:55.175898+0000","last_active":"2024-04-27T07:51:56.360560+0000","last_peered":"2024-04-27T07:51:56.360560+0000","last_clean":"2024-04-27T07:51:56.360560+0000","last_became_active":"2024-04-27T07:51:55.175061+0000","last_became_peered":"2024-04-27T07:51:55.175061+0000","last_unstale":"2024-04-27T07:51:56.360560+0000","last_undegraded":"2024-04-27T07:51:56.360560+0000","last_fullsized":"2024-04-27T07:51:56.360560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:12:16.083170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358682+0000","last_change":"2024-04-27T07:51:54.132734+0000","last_active":"2024-04-27T07:51:56.358682+0000","last_peered":"2024-04-27T07:51:56.358682+0000","last_clean":"2024-04-27T07:51:56.358682+0000","last_became_active":"2024-04-27T07:51:54.132374+0000","last_became_peered":"2024-04-27T07:51:54.132374+0000","last_unstale":"2024-04-27T07:51:56.358682+0000","last_undegraded":"2024-04-27T07:51:56.358682+0000","last_fullsized":"2024-04-27T07:51:56.358682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:47.730389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358236+0000","last_change":"2024-04-27T07:51:55.166684+0000","last_active":"2024-04-27T07:51:56.358236+0000","last_peered":"2024-04-27T07:51:56.358236+0000","last_clean":"2024-04-27T07:51:56.358236+0000","last_became_active":"2024-04-27T07:51:55.162281+0000","last_became_peered":"2024-04-27T07:51:55.162281+0000","last_unstale":"2024-04-27T07:51:56.358236+0000","last_undegraded":"2024-04-27T07:51:56.358236+0000","last_fullsized":"2024-04-27T07:51:56.358236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:29.864584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156758+0000","last_change":"2024-04-27T07:51:54.119029+0000","last_active":"2024-04-27T07:51:56.156758+0000","last_peered":"2024-04-27T07:51:56.156758+0000","last_clean":"2024-04-27T07:51:56.156758+0000","last_became_active":"2024-04-27T07:51:54.118506+0000","last_became_peered":"2024-04-27T07:51:54.118506+0000","last_unstale":"2024-04-27T07:51:56.156758+0000","last_undegraded":"2024-04-27T07:51:56.156758+0000","last_fullsized":"2024-04-27T07:51:56.156758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.856412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358165+0000","last_change":"2024-04-27T07:51:55.179381+0000","last_active":"2024-04-27T07:51:56.358165+0000","last_peered":"2024-04-27T07:51:56.358165+0000","last_clean":"2024-04-27T07:51:56.358165+0000","last_became_active":"2024-04-27T07:51:55.176583+0000","last_became_peered":"2024-04-27T07:51:55.176583+0000","last_unstale":"2024-04-27T07:51:56.358165+0000","last_undegraded":"2024-04-27T07:51:56.358165+0000","last_fullsized":"2024-04-27T07:51:56.358165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:51:19.108138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.362040+0000","last_change":"2024-04-27T07:51:54.134902+0000","last_active":"2024-04-27T07:51:56.362040+0000","last_peered":"2024-04-27T07:51:56.362040+0000","last_clean":"2024-04-27T07:51:56.362040+0000","last_became_active":"2024-04-27T07:51:54.134080+0000","last_became_peered":"2024-04-27T07:51:54.134080+0000","last_unstale":"2024-04-27T07:51:56.362040+0000","last_undegraded":"2024-04-27T07:51:56.362040+0000","last_fullsized":"2024-04-27T07:51:56.362040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:27.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361820+0000","last_change":"2024-04-27T07:51:55.166422+0000","last_active":"2024-04-27T07:51:56.361820+0000","last_peered":"2024-04-27T07:51:56.361820+0000","last_clean":"2024-04-27T07:51:56.361820+0000","last_became_active":"2024-04-27T07:51:55.164684+0000","last_became_peered":"2024-04-27T07:51:55.164684+0000","last_unstale":"2024-04-27T07:51:56.361820+0000","last_undegraded":"2024-04-27T07:51:56.361820+0000","last_fullsized":"2024-04-27T07:51:56.361820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:51:19.108138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358436+0000","last_change":"2024-04-27T07:51:54.111787+0000","last_active":"2024-04-27T07:51:56.358436+0000","last_peered":"2024-04-27T07:51:56.358436+0000","last_clean":"2024-04-27T07:51:56.358436+0000","last_became_active":"2024-04-27T07:51:54.111117+0000","last_became_peered":"2024-04-27T07:51:54.111117+0000","last_unstale":"2024-04-27T07:51:56.358436+0000","last_undegraded":"2024-04-27T07:51:56.358436+0000","last_fullsized":"2024-04-27T07:51:56.358436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:27.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163081+0000","last_change":"2024-04-27T07:51:55.184665+0000","last_active":"2024-04-27T07:51:56.163081+0000","last_peered":"2024-04-27T07:51:56.163081+0000","last_clean":"2024-04-27T07:51:56.163081+0000","last_became_active":"2024-04-27T07:51:55.184403+0000","last_became_peered":"2024-04-27T07:51:55.184403+0000","last_unstale":"2024-04-27T07:51:56.163081+0000","last_undegraded":"2024-04-27T07:51:56.163081+0000","last_fullsized":"2024-04-27T07:51:56.163081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:48.739020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158952+0000","last_change":"2024-04-27T07:51:54.138567+0000","last_active":"2024-04-27T07:51:56.158952+0000","last_peered":"2024-04-27T07:51:56.158952+0000","last_clean":"2024-04-27T07:51:56.158952+0000","last_became_active":"2024-04-27T07:51:54.137816+0000","last_became_peered":"2024-04-27T07:51:54.137816+0000","last_unstale":"2024-04-27T07:51:56.158952+0000","last_undegraded":"2024-04-27T07:51:56.158952+0000","last_fullsized":"2024-04-27T07:51:56.158952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:01:57.470600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358076+0000","last_change":"2024-04-27T07:51:55.171580+0000","last_active":"2024-04-27T07:51:56.358076+0000","last_peered":"2024-04-27T07:51:56.358076+0000","last_clean":"2024-04-27T07:51:56.358076+0000","last_became_active":"2024-04-27T07:51:55.168092+0000","last_became_peered":"2024-04-27T07:51:55.168092+0000","last_unstale":"2024-04-27T07:51:56.358076+0000","last_undegraded":"2024-04-27T07:51:56.358076+0000","last_fullsized":"2024-04-27T07:51:56.358076+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:16.089427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.162972+0000","last_change":"2024-04-27T07:51:54.125851+0000","last_active":"2024-04-27T07:51:56.162972+0000","last_peered":"2024-04-27T07:51:56.162972+0000","last_clean":"2024-04-27T07:51:56.162972+0000","last_became_active":"2024-04-27T07:51:54.125383+0000","last_became_peered":"2024-04-27T07:51:54.125383+0000","last_unstale":"2024-04-27T07:51:56.162972+0000","last_undegraded":"2024-04-27T07:51:56.162972+0000","last_fullsized":"2024-04-27T07:51:56.162972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361318+0000","last_change":"2024-04-27T07:51:55.206748+0000","last_active":"2024-04-27T07:51:56.361318+0000","last_peered":"2024-04-27T07:51:56.361318+0000","last_clean":"2024-04-27T07:51:56.361318+0000","last_became_active":"2024-04-27T07:51:55.205761+0000","last_became_peered":"2024-04-27T07:51:55.205761+0000","last_unstale":"2024-04-27T07:51:56.361318+0000","last_undegraded":"2024-04-27T07:51:56.361318+0000","last_fullsized":"2024-04-27T07:51:56.361318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:12:19.821551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357190+0000","last_change":"2024-04-27T07:51:54.126199+0000","last_active":"2024-04-27T07:51:56.357190+0000","last_peered":"2024-04-27T07:51:56.357190+0000","last_clean":"2024-04-27T07:51:56.357190+0000","last_became_active":"2024-04-27T07:51:54.125879+0000","last_became_peered":"2024-04-27T07:51:54.125879+0000","last_unstale":"2024-04-27T07:51:56.357190+0000","last_undegraded":"2024-04-27T07:51:56.357190+0000","last_fullsized":"2024-04-27T07:51:56.357190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:23:43.165438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360525+0000","last_change":"2024-04-27T07:51:55.194912+0000","last_active":"2024-04-27T07:51:56.360525+0000","last_peered":"2024-04-27T07:51:56.360525+0000","last_clean":"2024-04-27T07:51:56.360525+0000","last_became_active":"2024-04-27T07:51:55.193885+0000","last_became_peered":"2024-04-27T07:51:55.193885+0000","last_unstale":"2024-04-27T07:51:56.360525+0000","last_undegraded":"2024-04-27T07:51:56.360525+0000","last_fullsized":"2024-04-27T07:51:56.360525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:40.907109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357181+0000","last_change":"2024-04-27T07:51:54.113113+0000","last_active":"2024-04-27T07:51:56.357181+0000","last_peered":"2024-04-27T07:51:56.357181+0000","last_clean":"2024-04-27T07:51:56.357181+0000","last_became_active":"2024-04-27T07:51:54.112816+0000","last_became_peered":"2024-04-27T07:51:54.112816+0000","last_unstale":"2024-04-27T07:51:56.357181+0000","last_undegraded":"2024-04-27T07:51:56.357181+0000","last_fullsized":"2024-04-27T07:51:56.357181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:51:26.633386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159296+0000","last_change":"2024-04-27T07:51:55.190533+0000","last_active":"2024-04-27T07:51:56.159296+0000","last_peered":"2024-04-27T07:51:56.159296+0000","last_clean":"2024-04-27T07:51:56.159296+0000","last_became_active":"2024-04-27T07:51:55.189962+0000","last_became_peered":"2024-04-27T07:51:55.189962+0000","last_unstale":"2024-04-27T07:51:56.159296+0000","last_undegraded":"2024-04-27T07:51:56.159296+0000","last_fullsized":"2024-04-27T07:51:56.159296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:30.207284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163680+0000","last_change":"2024-04-27T07:51:54.121271+0000","last_active":"2024-04-27T07:51:56.163680+0000","last_peered":"2024-04-27T07:51:56.163680+0000","last_clean":"2024-04-27T07:51:56.163680+0000","last_became_active":"2024-04-27T07:51:54.120834+0000","last_became_peered":"2024-04-27T07:51:54.120834+0000","last_unstale":"2024-04-27T07:51:56.163680+0000","last_undegraded":"2024-04-27T07:51:56.163680+0000","last_fullsized":"2024-04-27T07:51:56.163680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:08.726549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360699+0000","last_change":"2024-04-27T07:51:55.194049+0000","last_active":"2024-04-27T07:51:56.360699+0000","last_peered":"2024-04-27T07:51:56.360699+0000","last_clean":"2024-04-27T07:51:56.360699+0000","last_became_active":"2024-04-27T07:51:55.193139+0000","last_became_peered":"2024-04-27T07:51:55.193139+0000","last_unstale":"2024-04-27T07:51:56.360699+0000","last_undegraded":"2024-04-27T07:51:56.360699+0000","last_fullsized":"2024-04-27T07:51:56.360699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:51:36.231986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159252+0000","last_change":"2024-04-27T07:51:54.144802+0000","last_active":"2024-04-27T07:51:56.159252+0000","last_peered":"2024-04-27T07:51:56.159252+0000","last_clean":"2024-04-27T07:51:56.159252+0000","last_became_active":"2024-04-27T07:51:54.144063+0000","last_became_peered":"2024-04-27T07:51:54.144063+0000","last_unstale":"2024-04-27T07:51:56.159252+0000","last_undegraded":"2024-04-27T07:51:56.159252+0000","last_fullsized":"2024-04-27T07:51:56.159252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:25.687567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163495+0000","last_change":"2024-04-27T07:51:55.197786+0000","last_active":"2024-04-27T07:51:56.163495+0000","last_peered":"2024-04-27T07:51:56.163495+0000","last_clean":"2024-04-27T07:51:56.163495+0000","last_became_active":"2024-04-27T07:51:55.197271+0000","last_became_peered":"2024-04-27T07:51:55.197271+0000","last_unstale":"2024-04-27T07:51:56.163495+0000","last_undegraded":"2024-04-27T07:51:56.163495+0000","last_fullsized":"2024-04-27T07:51:56.163495+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:23:20.564940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159284+0000","last_change":"2024-04-27T07:51:54.144467+0000","last_active":"2024-04-27T07:51:56.159284+0000","last_peered":"2024-04-27T07:51:56.159284+0000","last_clean":"2024-04-27T07:51:56.159284+0000","last_became_active":"2024-04-27T07:51:54.143451+0000","last_became_peered":"2024-04-27T07:51:54.143451+0000","last_unstale":"2024-04-27T07:51:56.159284+0000","last_undegraded":"2024-04-27T07:51:56.159284+0000","last_fullsized":"2024-04-27T07:51:56.159284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:00:05.467783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155125+0000","last_change":"2024-04-27T07:51:55.163330+0000","last_active":"2024-04-27T07:51:56.155125+0000","last_peered":"2024-04-27T07:51:56.155125+0000","last_clean":"2024-04-27T07:51:56.155125+0000","last_became_active":"2024-04-27T07:51:55.163003+0000","last_became_peered":"2024-04-27T07:51:55.163003+0000","last_unstale":"2024-04-27T07:51:56.155125+0000","last_undegraded":"2024-04-27T07:51:56.155125+0000","last_fullsized":"2024-04-27T07:51:56.155125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:16:47.348161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158988+0000","last_change":"2024-04-27T07:51:54.139192+0000","last_active":"2024-04-27T07:51:56.158988+0000","last_peered":"2024-04-27T07:51:56.158988+0000","last_clean":"2024-04-27T07:51:56.158988+0000","last_became_active":"2024-04-27T07:51:54.137084+0000","last_became_peered":"2024-04-27T07:51:54.137084+0000","last_unstale":"2024-04-27T07:51:56.158988+0000","last_undegraded":"2024-04-27T07:51:56.158988+0000","last_fullsized":"2024-04-27T07:51:56.158988+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:17:11.463660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154952+0000","last_change":"2024-04-27T07:51:55.191255+0000","last_active":"2024-04-27T07:51:56.154952+0000","last_peered":"2024-04-27T07:51:56.154952+0000","last_clean":"2024-04-27T07:51:56.154952+0000","last_became_active":"2024-04-27T07:51:55.190911+0000","last_became_peered":"2024-04-27T07:51:55.190911+0000","last_unstale":"2024-04-27T07:51:56.154952+0000","last_undegraded":"2024-04-27T07:51:56.154952+0000","last_fullsized":"2024-04-27T07:51:56.154952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:48:31.253337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156464+0000","last_change":"2024-04-27T07:51:54.117841+0000","last_active":"2024-04-27T07:51:56.156464+0000","last_peered":"2024-04-27T07:51:56.156464+0000","last_clean":"2024-04-27T07:51:56.156464+0000","last_became_active":"2024-04-27T07:51:54.117462+0000","last_became_peered":"2024-04-27T07:51:54.117462+0000","last_unstale":"2024-04-27T07:51:56.156464+0000","last_undegraded":"2024-04-27T07:51:56.156464+0000","last_fullsized":"2024-04-27T07:51:56.156464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:50.409799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155202+0000","last_change":"2024-04-27T07:51:55.166521+0000","last_active":"2024-04-27T07:51:56.155202+0000","last_peered":"2024-04-27T07:51:56.155202+0000","last_clean":"2024-04-27T07:51:56.155202+0000","last_became_active":"2024-04-27T07:51:55.166043+0000","last_became_peered":"2024-04-27T07:51:55.166043+0000","last_unstale":"2024-04-27T07:51:56.155202+0000","last_undegraded":"2024-04-27T07:51:56.155202+0000","last_fullsized":"2024-04-27T07:51:56.155202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:54.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359351+0000","last_change":"2024-04-27T07:51:54.132661+0000","last_active":"2024-04-27T07:51:56.359351+0000","last_peered":"2024-04-27T07:51:56.359351+0000","last_clean":"2024-04-27T07:51:56.359351+0000","last_became_active":"2024-04-27T07:51:54.131852+0000","last_became_peered":"2024-04-27T07:51:54.131852+0000","last_unstale":"2024-04-27T07:51:56.359351+0000","last_undegraded":"2024-04-27T07:51:56.359351+0000","last_fullsized":"2024-04-27T07:51:56.359351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:21.051122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154514+0000","last_change":"2024-04-27T07:51:55.164715+0000","last_active":"2024-04-27T07:51:56.154514+0000","last_peered":"2024-04-27T07:51:56.154514+0000","last_clean":"2024-04-27T07:51:56.154514+0000","last_became_active":"2024-04-27T07:51:55.164419+0000","last_became_peered":"2024-04-27T07:51:55.164419+0000","last_unstale":"2024-04-27T07:51:56.154514+0000","last_undegraded":"2024-04-27T07:51:56.154514+0000","last_fullsized":"2024-04-27T07:51:56.154514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:38.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157994+0000","last_change":"2024-04-27T07:51:54.136947+0000","last_active":"2024-04-27T07:51:56.157994+0000","last_peered":"2024-04-27T07:51:56.157994+0000","last_clean":"2024-04-27T07:51:56.157994+0000","last_became_active":"2024-04-27T07:51:54.131725+0000","last_became_peered":"2024-04-27T07:51:54.131725+0000","last_unstale":"2024-04-27T07:51:56.157994+0000","last_undegraded":"2024-04-27T07:51:56.157994+0000","last_fullsized":"2024-04-27T07:51:56.157994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:32.938716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359219+0000","last_change":"2024-04-27T07:51:55.187793+0000","last_active":"2024-04-27T07:51:56.359219+0000","last_peered":"2024-04-27T07:51:56.359219+0000","last_clean":"2024-04-27T07:51:56.359219+0000","last_became_active":"2024-04-27T07:51:55.185683+0000","last_became_peered":"2024-04-27T07:51:55.185683+0000","last_unstale":"2024-04-27T07:51:56.359219+0000","last_undegraded":"2024-04-27T07:51:56.359219+0000","last_fullsized":"2024-04-27T07:51:56.359219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:33:02.075078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359570+0000","last_change":"2024-04-27T07:51:54.134411+0000","last_active":"2024-04-27T07:51:56.359570+0000","last_peered":"2024-04-27T07:51:56.359570+0000","last_clean":"2024-04-27T07:51:56.359570+0000","last_became_active":"2024-04-27T07:51:54.133697+0000","last_became_peered":"2024-04-27T07:51:54.133697+0000","last_unstale":"2024-04-27T07:51:56.359570+0000","last_undegraded":"2024-04-27T07:51:56.359570+0000","last_fullsized":"2024-04-27T07:51:56.359570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:13.457940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154799+0000","last_change":"2024-04-27T07:51:55.172934+0000","last_active":"2024-04-27T07:51:56.154799+0000","last_peered":"2024-04-27T07:51:56.154799+0000","last_clean":"2024-04-27T07:51:56.154799+0000","last_became_active":"2024-04-27T07:51:55.172600+0000","last_became_peered":"2024-04-27T07:51:55.172600+0000","last_unstale":"2024-04-27T07:51:56.154799+0000","last_undegraded":"2024-04-27T07:51:56.154799+0000","last_fullsized":"2024-04-27T07:51:56.154799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:09.735179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163760+0000","last_change":"2024-04-27T07:51:54.122872+0000","last_active":"2024-04-27T07:51:56.163760+0000","last_peered":"2024-04-27T07:51:56.163760+0000","last_clean":"2024-04-27T07:51:56.163760+0000","last_became_active":"2024-04-27T07:51:54.122391+0000","last_became_peered":"2024-04-27T07:51:54.122391+0000","last_unstale":"2024-04-27T07:51:56.163760+0000","last_undegraded":"2024-04-27T07:51:56.163760+0000","last_fullsized":"2024-04-27T07:51:56.163760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:13.052684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360823+0000","last_change":"2024-04-27T07:51:55.164723+0000","last_active":"2024-04-27T07:51:56.360823+0000","last_peered":"2024-04-27T07:51:56.360823+0000","last_clean":"2024-04-27T07:51:56.360823+0000","last_became_active":"2024-04-27T07:51:55.163877+0000","last_became_peered":"2024-04-27T07:51:55.163877+0000","last_unstale":"2024-04-27T07:51:56.360823+0000","last_undegraded":"2024-04-27T07:51:56.360823+0000","last_fullsized":"2024-04-27T07:51:56.360823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:32.692411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.161937+0000","last_change":"2024-04-27T07:51:54.112256+0000","last_active":"2024-04-27T07:51:56.161937+0000","last_peered":"2024-04-27T07:51:56.161937+0000","last_clean":"2024-04-27T07:51:56.161937+0000","last_became_active":"2024-04-27T07:51:54.111768+0000","last_became_peered":"2024-04-27T07:51:54.111768+0000","last_unstale":"2024-04-27T07:51:56.161937+0000","last_undegraded":"2024-04-27T07:51:56.161937+0000","last_fullsized":"2024-04-27T07:51:56.161937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156691+0000","last_change":"2024-04-27T07:51:55.163718+0000","last_active":"2024-04-27T07:51:56.156691+0000","last_peered":"2024-04-27T07:51:56.156691+0000","last_clean":"2024-04-27T07:51:56.156691+0000","last_became_active":"2024-04-27T07:51:55.163432+0000","last_became_peered":"2024-04-27T07:51:55.163432+0000","last_unstale":"2024-04-27T07:51:56.156691+0000","last_undegraded":"2024-04-27T07:51:56.156691+0000","last_fullsized":"2024-04-27T07:51:56.156691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:57.706999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163543+0000","last_change":"2024-04-27T07:51:54.108254+0000","last_active":"2024-04-27T07:51:56.163543+0000","last_peered":"2024-04-27T07:51:56.163543+0000","last_clean":"2024-04-27T07:51:56.163543+0000","last_became_active":"2024-04-27T07:51:54.107946+0000","last_became_peered":"2024-04-27T07:51:54.107946+0000","last_unstale":"2024-04-27T07:51:56.163543+0000","last_undegraded":"2024-04-27T07:51:56.163543+0000","last_fullsized":"2024-04-27T07:51:56.163543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:50.409799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359367+0000","last_change":"2024-04-27T07:51:55.181437+0000","last_active":"2024-04-27T07:51:56.359367+0000","last_peered":"2024-04-27T07:51:56.359367+0000","last_clean":"2024-04-27T07:51:56.359367+0000","last_became_active":"2024-04-27T07:51:55.180517+0000","last_became_peered":"2024-04-27T07:51:55.180517+0000","last_unstale":"2024-04-27T07:51:56.359367+0000","last_undegraded":"2024-04-27T07:51:56.359367+0000","last_fullsized":"2024-04-27T07:51:56.359367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:32:07.108034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156679+0000","last_change":"2024-04-27T07:51:54.120139+0000","last_active":"2024-04-27T07:51:56.156679+0000","last_peered":"2024-04-27T07:51:56.156679+0000","last_clean":"2024-04-27T07:51:56.156679+0000","last_became_active":"2024-04-27T07:51:54.119538+0000","last_became_peered":"2024-04-27T07:51:54.119538+0000","last_unstale":"2024-04-27T07:51:56.156679+0000","last_undegraded":"2024-04-27T07:51:56.156679+0000","last_fullsized":"2024-04-27T07:51:56.156679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:08.726549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156542+0000","last_change":"2024-04-27T07:51:55.169350+0000","last_active":"2024-04-27T07:51:56.156542+0000","last_peered":"2024-04-27T07:51:56.156542+0000","last_clean":"2024-04-27T07:51:56.156542+0000","last_became_active":"2024-04-27T07:51:55.169006+0000","last_became_peered":"2024-04-27T07:51:55.169006+0000","last_unstale":"2024-04-27T07:51:56.156542+0000","last_undegraded":"2024-04-27T07:51:56.156542+0000","last_fullsized":"2024-04-27T07:51:56.156542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:51:27.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163057+0000","last_change":"2024-04-27T07:51:54.125639+0000","last_active":"2024-04-27T07:51:56.163057+0000","last_peered":"2024-04-27T07:51:56.163057+0000","last_clean":"2024-04-27T07:51:56.163057+0000","last_became_active":"2024-04-27T07:51:54.125374+0000","last_became_peered":"2024-04-27T07:51:54.125374+0000","last_unstale":"2024-04-27T07:51:56.163057+0000","last_undegraded":"2024-04-27T07:51:56.163057+0000","last_fullsized":"2024-04-27T07:51:56.163057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.856412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357201+0000","last_change":"2024-04-27T07:51:55.157964+0000","last_active":"2024-04-27T07:51:56.357201+0000","last_peered":"2024-04-27T07:51:56.357201+0000","last_clean":"2024-04-27T07:51:56.357201+0000","last_became_active":"2024-04-27T07:51:55.157519+0000","last_became_peered":"2024-04-27T07:51:55.157519+0000","last_unstale":"2024-04-27T07:51:56.357201+0000","last_undegraded":"2024-04-27T07:51:56.357201+0000","last_fullsized":"2024-04-27T07:51:56.357201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:12:16.083170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158133+0000","last_change":"2024-04-27T07:51:54.144370+0000","last_active":"2024-04-27T07:51:56.158133+0000","last_peered":"2024-04-27T07:51:56.158133+0000","last_clean":"2024-04-27T07:51:56.158133+0000","last_became_active":"2024-04-27T07:51:54.143612+0000","last_became_peered":"2024-04-27T07:51:54.143612+0000","last_unstale":"2024-04-27T07:51:56.158133+0000","last_undegraded":"2024-04-27T07:51:56.158133+0000","last_fullsized":"2024-04-27T07:51:56.158133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:44:55.866398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158084+0000","last_change":"2024-04-27T07:51:55.192563+0000","last_active":"2024-04-27T07:51:56.158084+0000","last_peered":"2024-04-27T07:51:56.158084+0000","last_clean":"2024-04-27T07:51:56.158084+0000","last_became_active":"2024-04-27T07:51:55.191806+0000","last_became_peered":"2024-04-27T07:51:55.191806+0000","last_unstale":"2024-04-27T07:51:56.158084+0000","last_undegraded":"2024-04-27T07:51:56.158084+0000","last_fullsized":"2024-04-27T07:51:56.158084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:14.466571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357462+0000","last_change":"2024-04-27T07:51:54.132729+0000","last_active":"2024-04-27T07:51:56.357462+0000","last_peered":"2024-04-27T07:51:56.357462+0000","last_clean":"2024-04-27T07:51:56.357462+0000","last_became_active":"2024-04-27T07:51:54.132309+0000","last_became_peered":"2024-04-27T07:51:54.132309+0000","last_unstale":"2024-04-27T07:51:56.357462+0000","last_undegraded":"2024-04-27T07:51:56.357462+0000","last_fullsized":"2024-04-27T07:51:56.357462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:13.457940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360155+0000","last_change":"2024-04-27T07:51:55.186929+0000","last_active":"2024-04-27T07:51:56.360155+0000","last_peered":"2024-04-27T07:51:56.360155+0000","last_clean":"2024-04-27T07:51:56.360155+0000","last_became_active":"2024-04-27T07:51:55.176976+0000","last_became_peered":"2024-04-27T07:51:55.176976+0000","last_unstale":"2024-04-27T07:51:56.360155+0000","last_undegraded":"2024-04-27T07:51:56.360155+0000","last_fullsized":"2024-04-27T07:51:56.360155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:29:07.528389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360821+0000","last_change":"2024-04-27T07:51:54.146133+0000","last_active":"2024-04-27T07:51:56.360821+0000","last_peered":"2024-04-27T07:51:56.360821+0000","last_clean":"2024-04-27T07:51:56.360821+0000","last_became_active":"2024-04-27T07:51:54.145807+0000","last_became_peered":"2024-04-27T07:51:54.145807+0000","last_unstale":"2024-04-27T07:51:56.360821+0000","last_undegraded":"2024-04-27T07:51:56.360821+0000","last_fullsized":"2024-04-27T07:51:56.360821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:51:32.021029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359505+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:51:56.359505+0000","last_peered":"2024-04-27T07:51:56.359505+0000","last_clean":"2024-04-27T07:51:56.359505+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:51:56.359505+0000","last_undegraded":"2024-04-27T07:51:56.359505+0000","last_fullsized":"2024-04-27T07:51:56.359505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":740295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":149941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":151471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":385024,"data_stored":149907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":151471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":740205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:51:59.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:51:59.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.374+0000 7f2a45f0e640 1 Processor -- start 2024-04-27T07:51:59.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.374+0000 7f2a45f0e640 1 -- start start 2024-04-27T07:51:59.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a45f0e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2a400b19f0 0x7f2a4015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a45f0e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a4015e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a45f0e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400b1ff0 con 0x7f2a4015c970 2024-04-27T07:51:59.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a45f0e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2a400cf130 con 0x7f2a400b19f0 2024-04-27T07:51:59.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a45f0e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400ce090 con 0x7f2a4015e030 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2a4015c970 0x7f2a400b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50564/0 (socket says 172.21.15.22:50564) 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 -- 172.21.15.22:0/3800930235 learned_addr learned my addr 172.21.15.22:0/3800930235 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a4015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3effd640 1 --2- 172.21.15.22:0/3800930235 >> v2:172.21.15.183:3300/0 conn(0x7f2a400b19f0 0x7f2a4015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956397162 0 0) 0x7f2a400b1ff0 con 0x7f2a4015c970 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a28003530 con 0x7f2a4015c970 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4274819101 0 0) 0x7f2a28003530 con 0x7f2a4015c970 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 msgr2=0x7f2a4015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a4015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 >> v2:172.21.15.183:3300/0 conn(0x7f2a400b19f0 msgr2=0x7f2a4015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 --2- 172.21.15.22:0/3800930235 >> v2:172.21.15.183:3300/0 conn(0x7f2a400b19f0 0x7f2a4015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400ce370 con 0x7f2a4015c970 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f2a38001a20 con 0x7f2a4015c970 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 >> v1:172.21.15.22:6789/0 conn(0x7f2a4015c970 legacy=0x7f2a400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a280039c0 0x7f2a28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a28004810 0x7f2a28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400ce090 con 0x7f2a280039c0 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2800b9d0 con 0x7f2a28004810 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2800bb80 con 0x7f2a4015e030 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a280039c0 0x7f2a28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3effd640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a28004810 0x7f2a28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 -- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 msgr2=0x7f2a280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 -- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a28004810 msgr2=0x7f2a28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a28004810 0x7f2a28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3f7fe640 1 -- 172.21.15.22:0/3800930235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a28003530 con 0x7f2a280039c0 2024-04-27T07:51:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.378+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a4015e030 0x7f2a280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:51:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a280039c0 0x7f2a28003de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800a880 tx=0x7f2a3801d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da00012d04a0e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800a4d0 con 0x7f2a280039c0 2024-04-27T07:51:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a380ad630 con 0x7f2a280039c0 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3e7fc640 1 -- 172.21.15.22:0/3800930235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380ada20 con 0x7f2a280039c0 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3800930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a280039c0 msgr2=0x7f2a28003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3800930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a280039c0 0x7f2a28003de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800a880 tx=0x7f2a3801d320 comp rx=0 tx=0).stop 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 -- 172.21.15.22:0/3800930235 reap_dead start 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3800930235 shutdown_connections 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3800930235 >> 172.21.15.22:0/3800930235 conn(0x7f2a400b68d0 msgr2=0x7f2a400b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3800930235 shutdown_connections 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3800930235 wait complete. 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 Processor -- start 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- start start 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 0x7f2a400b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a401588a0 0x7f2a401579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 0x7f2a4014bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400b10a0 con 0x7f2a40164a60 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400b0f20 con 0x7f2a401588a0 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a45f0e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4015ec50 con 0x7f2a280039c0 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 0x7f2a400b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 0x7f2a4014bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 0x7f2a400b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:58054/0 (socket says 172.21.15.22:58054) 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3f7fe640 1 -- 172.21.15.22:0/3272633081 learned_addr learned my addr 172.21.15.22:0/3272633081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3effd640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a401588a0 0x7f2a401579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 -- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 msgr2=0x7f2a400b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 0x7f2a400b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 -- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a401588a0 msgr2=0x7f2a401579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a401588a0 0x7f2a401579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.382+0000 7f2a3ffff640 1 -- 172.21.15.22:0/3272633081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40157f20 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a3ffff640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 0x7f2a4014bca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800bd50 tx=0x7f2a30010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000f040 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3000e8a0 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a400b4b40 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ec70 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401571e0 con 0x7f2a40164a60 2024-04-27T07:51:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a40069340 con 0x7f2a40164a60 2024-04-27T07:51:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2a3001c400 con 0x7f2a40164a60 2024-04-27T07:51:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 0x7f2a2c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2a30086f80 con 0x7f2a40164a60 2024-04-27T07:51:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.386+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 0x7f2a2c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.390+0000 7f2a3f7fe640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 0x7f2a2c06a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001d20 tx=0x7f2a38007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.390+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2a30018030 con 0x7f2a40164a60 2024-04-27T07:51:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.546+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a4015f3e0 con 0x7f2a2c0683b0 2024-04-27T07:51:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.562+0000 7f2a27fff640 1 -- 172.21.15.22:0/3272633081 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f2a4015f3e0 con 0x7f2a2c0683b0 2024-04-27T07:51:59.568 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:51:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:51:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 msgr2=0x7f2a2c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 0x7f2a2c06a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001d20 tx=0x7f2a38007040 comp rx=0 tx=0).stop 2024-04-27T07:51:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 msgr2=0x7f2a4014bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 0x7f2a4014bca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800bd50 tx=0x7f2a30010c20 comp rx=0 tx=0).stop 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 shutdown_connections 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2a2c0683b0 0x7f2a2c06a870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a40164a60 0x7f2a4014bca0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a401588a0 0x7f2a401579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 --2- 172.21.15.22:0/3272633081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a280039c0 0x7f2a400b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 >> 172.21.15.22:0/3272633081 conn(0x7f2a400b68d0 msgr2=0x7f2a40156770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 shutdown_connections 2024-04-27T07:51:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.566+0000 7f2a45f0e640 1 -- 172.21.15.22:0/3272633081 wait complete. 2024-04-27T07:51:59.585 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-27T07:51:58.171469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241176,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733544,"statfs":{"total":773094113280,"available":772847149056,"internally_reserved":0,"allocated":4300800,"data_stored":2384252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078600"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361085+0000","last_change":"2024-04-27T07:51:55.188412+0000","last_active":"2024-04-27T07:51:56.361085+0000","last_peered":"2024-04-27T07:51:56.361085+0000","last_clean":"2024-04-27T07:51:56.361085+0000","last_became_active":"2024-04-27T07:51:55.186996+0000","last_became_peered":"2024-04-27T07:51:55.186996+0000","last_unstale":"2024-04-27T07:51:56.361085+0000","last_undegraded":"2024-04-27T07:51:56.361085+0000","last_fullsized":"2024-04-27T07:51:56.361085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:49.764377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358126+0000","last_change":"2024-04-27T07:51:55.172585+0000","last_active":"2024-04-27T07:51:56.358126+0000","last_peered":"2024-04-27T07:51:56.358126+0000","last_clean":"2024-04-27T07:51:56.358126+0000","last_became_active":"2024-04-27T07:51:55.172067+0000","last_became_peered":"2024-04-27T07:51:55.172067+0000","last_unstale":"2024-04-27T07:51:56.358126+0000","last_undegraded":"2024-04-27T07:51:56.358126+0000","last_fullsized":"2024-04-27T07:51:56.358126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:05:35.477534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361914+0000","last_change":"2024-04-27T07:51:55.167000+0000","last_active":"2024-04-27T07:51:56.361914+0000","last_peered":"2024-04-27T07:51:56.361914+0000","last_clean":"2024-04-27T07:51:56.361914+0000","last_became_active":"2024-04-27T07:51:55.165675+0000","last_became_peered":"2024-04-27T07:51:55.165675+0000","last_unstale":"2024-04-27T07:51:56.361914+0000","last_undegraded":"2024-04-27T07:51:56.361914+0000","last_fullsized":"2024-04-27T07:51:56.361914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:29.864584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163375+0000","last_change":"2024-04-27T07:51:55.197784+0000","last_active":"2024-04-27T07:51:56.163375+0000","last_peered":"2024-04-27T07:51:56.163375+0000","last_clean":"2024-04-27T07:51:56.163375+0000","last_became_active":"2024-04-27T07:51:55.197481+0000","last_became_peered":"2024-04-27T07:51:55.197481+0000","last_unstale":"2024-04-27T07:51:56.163375+0000","last_undegraded":"2024-04-27T07:51:56.163375+0000","last_fullsized":"2024-04-27T07:51:56.163375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:31.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154687+0000","last_change":"2024-04-27T07:51:55.172087+0000","last_active":"2024-04-27T07:51:56.154687+0000","last_peered":"2024-04-27T07:51:56.154687+0000","last_clean":"2024-04-27T07:51:56.154687+0000","last_became_active":"2024-04-27T07:51:55.171808+0000","last_became_peered":"2024-04-27T07:51:55.171808+0000","last_unstale":"2024-04-27T07:51:56.154687+0000","last_undegraded":"2024-04-27T07:51:56.154687+0000","last_fullsized":"2024-04-27T07:51:56.154687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:07.865042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357667+0000","last_change":"2024-04-27T07:51:55.154885+0000","last_active":"2024-04-27T07:51:56.357667+0000","last_peered":"2024-04-27T07:51:56.357667+0000","last_clean":"2024-04-27T07:51:56.357667+0000","last_became_active":"2024-04-27T07:51:55.154369+0000","last_became_peered":"2024-04-27T07:51:55.154369+0000","last_unstale":"2024-04-27T07:51:56.357667+0000","last_undegraded":"2024-04-27T07:51:56.357667+0000","last_fullsized":"2024-04-27T07:51:56.357667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:51:36.231986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359562+0000","last_change":"2024-04-27T07:51:55.186506+0000","last_active":"2024-04-27T07:51:56.359562+0000","last_peered":"2024-04-27T07:51:56.359562+0000","last_clean":"2024-04-27T07:51:56.359562+0000","last_became_active":"2024-04-27T07:51:55.184683+0000","last_became_peered":"2024-04-27T07:51:55.184683+0000","last_unstale":"2024-04-27T07:51:56.359562+0000","last_undegraded":"2024-04-27T07:51:56.359562+0000","last_fullsized":"2024-04-27T07:51:56.359562+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:31.927909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358783+0000","last_change":"2024-04-27T07:51:55.205178+0000","last_active":"2024-04-27T07:51:56.358783+0000","last_peered":"2024-04-27T07:51:56.358783+0000","last_clean":"2024-04-27T07:51:56.358783+0000","last_became_active":"2024-04-27T07:51:55.204662+0000","last_became_peered":"2024-04-27T07:51:55.204662+0000","last_unstale":"2024-04-27T07:51:56.358783+0000","last_undegraded":"2024-04-27T07:51:56.358783+0000","last_fullsized":"2024-04-27T07:51:56.358783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:38.267349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159665+0000","last_change":"2024-04-27T07:51:55.190539+0000","last_active":"2024-04-27T07:51:56.159665+0000","last_peered":"2024-04-27T07:51:56.159665+0000","last_clean":"2024-04-27T07:51:56.159665+0000","last_became_active":"2024-04-27T07:51:55.189855+0000","last_became_peered":"2024-04-27T07:51:55.189855+0000","last_unstale":"2024-04-27T07:51:56.159665+0000","last_undegraded":"2024-04-27T07:51:56.159665+0000","last_fullsized":"2024-04-27T07:51:56.159665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:23:20.564940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163390+0000","last_change":"2024-04-27T07:51:55.167873+0000","last_active":"2024-04-27T07:51:56.163390+0000","last_peered":"2024-04-27T07:51:56.163390+0000","last_clean":"2024-04-27T07:51:56.163390+0000","last_became_active":"2024-04-27T07:51:55.167578+0000","last_became_peered":"2024-04-27T07:51:55.167578+0000","last_unstale":"2024-04-27T07:51:56.163390+0000","last_undegraded":"2024-04-27T07:51:56.163390+0000","last_fullsized":"2024-04-27T07:51:56.163390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:30.207284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361376+0000","last_change":"2024-04-27T07:51:55.185096+0000","last_active":"2024-04-27T07:51:56.361376+0000","last_peered":"2024-04-27T07:51:56.361376+0000","last_clean":"2024-04-27T07:51:56.361376+0000","last_became_active":"2024-04-27T07:51:55.164596+0000","last_became_peered":"2024-04-27T07:51:55.164596+0000","last_unstale":"2024-04-27T07:51:56.361376+0000","last_undegraded":"2024-04-27T07:51:56.361376+0000","last_fullsized":"2024-04-27T07:51:56.361376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:09:45.090148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361188+0000","last_change":"2024-04-27T07:51:55.175952+0000","last_active":"2024-04-27T07:51:56.361188+0000","last_peered":"2024-04-27T07:51:56.361188+0000","last_clean":"2024-04-27T07:51:56.361188+0000","last_became_active":"2024-04-27T07:51:55.174964+0000","last_became_peered":"2024-04-27T07:51:55.174964+0000","last_unstale":"2024-04-27T07:51:56.361188+0000","last_undegraded":"2024-04-27T07:51:56.361188+0000","last_fullsized":"2024-04-27T07:51:56.361188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:05:35.477534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154879+0000","last_change":"2024-04-27T07:51:55.163174+0000","last_active":"2024-04-27T07:51:56.154879+0000","last_peered":"2024-04-27T07:51:56.154879+0000","last_clean":"2024-04-27T07:51:56.154879+0000","last_became_active":"2024-04-27T07:51:55.162870+0000","last_became_peered":"2024-04-27T07:51:55.162870+0000","last_unstale":"2024-04-27T07:51:56.154879+0000","last_undegraded":"2024-04-27T07:51:56.154879+0000","last_fullsized":"2024-04-27T07:51:56.154879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:14.061315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357731+0000","last_change":"2024-04-27T07:51:55.171896+0000","last_active":"2024-04-27T07:51:56.357731+0000","last_peered":"2024-04-27T07:51:56.357731+0000","last_clean":"2024-04-27T07:51:56.357731+0000","last_became_active":"2024-04-27T07:51:55.171589+0000","last_became_peered":"2024-04-27T07:51:55.171589+0000","last_unstale":"2024-04-27T07:51:56.357731+0000","last_undegraded":"2024-04-27T07:51:56.357731+0000","last_fullsized":"2024-04-27T07:51:56.357731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:32.692411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360283+0000","last_change":"2024-04-27T07:51:55.169407+0000","last_active":"2024-04-27T07:51:56.360283+0000","last_peered":"2024-04-27T07:51:56.360283+0000","last_clean":"2024-04-27T07:51:56.360283+0000","last_became_active":"2024-04-27T07:51:55.168695+0000","last_became_peered":"2024-04-27T07:51:55.168695+0000","last_unstale":"2024-04-27T07:51:56.360283+0000","last_undegraded":"2024-04-27T07:51:56.360283+0000","last_fullsized":"2024-04-27T07:51:56.360283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:20.305883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359370+0000","last_change":"2024-04-27T07:51:55.207162+0000","last_active":"2024-04-27T07:51:56.359370+0000","last_peered":"2024-04-27T07:51:56.359370+0000","last_clean":"2024-04-27T07:51:56.359370+0000","last_became_active":"2024-04-27T07:51:55.206324+0000","last_became_peered":"2024-04-27T07:51:55.206324+0000","last_unstale":"2024-04-27T07:51:56.359370+0000","last_undegraded":"2024-04-27T07:51:56.359370+0000","last_fullsized":"2024-04-27T07:51:56.359370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:38:46.584110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361689+0000","last_change":"2024-04-27T07:51:55.179158+0000","last_active":"2024-04-27T07:51:56.361689+0000","last_peered":"2024-04-27T07:51:56.361689+0000","last_clean":"2024-04-27T07:51:56.361689+0000","last_became_active":"2024-04-27T07:51:55.177884+0000","last_became_peered":"2024-04-27T07:51:55.177884+0000","last_unstale":"2024-04-27T07:51:56.361689+0000","last_undegraded":"2024-04-27T07:51:56.361689+0000","last_fullsized":"2024-04-27T07:51:56.361689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:38.648029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358331+0000","last_change":"2024-04-27T07:51:55.159436+0000","last_active":"2024-04-27T07:51:56.358331+0000","last_peered":"2024-04-27T07:51:56.358331+0000","last_clean":"2024-04-27T07:51:56.358331+0000","last_became_active":"2024-04-27T07:51:55.159137+0000","last_became_peered":"2024-04-27T07:51:55.159137+0000","last_unstale":"2024-04-27T07:51:56.358331+0000","last_undegraded":"2024-04-27T07:51:56.358331+0000","last_fullsized":"2024-04-27T07:51:56.358331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:59:32.806059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357906+0000","last_change":"2024-04-27T07:51:55.165966+0000","last_active":"2024-04-27T07:51:56.357906+0000","last_peered":"2024-04-27T07:51:56.357906+0000","last_clean":"2024-04-27T07:51:56.357906+0000","last_became_active":"2024-04-27T07:51:55.162244+0000","last_became_peered":"2024-04-27T07:51:55.162244+0000","last_unstale":"2024-04-27T07:51:56.357906+0000","last_undegraded":"2024-04-27T07:51:56.357906+0000","last_fullsized":"2024-04-27T07:51:56.357906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:34.465556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154683+0000","last_change":"2024-04-27T07:51:55.172770+0000","last_active":"2024-04-27T07:51:56.154683+0000","last_peered":"2024-04-27T07:51:56.154683+0000","last_clean":"2024-04-27T07:51:56.154683+0000","last_became_active":"2024-04-27T07:51:55.172473+0000","last_became_peered":"2024-04-27T07:51:55.172473+0000","last_unstale":"2024-04-27T07:51:56.154683+0000","last_undegraded":"2024-04-27T07:51:56.154683+0000","last_fullsized":"2024-04-27T07:51:56.154683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:51.418430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158599+0000","last_change":"2024-04-27T07:51:55.191317+0000","last_active":"2024-04-27T07:51:56.158599+0000","last_peered":"2024-04-27T07:51:56.158599+0000","last_clean":"2024-04-27T07:51:56.158599+0000","last_became_active":"2024-04-27T07:51:55.190428+0000","last_became_peered":"2024-04-27T07:51:55.190428+0000","last_unstale":"2024-04-27T07:51:56.158599+0000","last_undegraded":"2024-04-27T07:51:56.158599+0000","last_fullsized":"2024-04-27T07:51:56.158599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:48.739020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158129+0000","last_change":"2024-04-27T07:51:55.192573+0000","last_active":"2024-04-27T07:51:56.158129+0000","last_peered":"2024-04-27T07:51:56.158129+0000","last_clean":"2024-04-27T07:51:56.158129+0000","last_became_active":"2024-04-27T07:51:55.191795+0000","last_became_peered":"2024-04-27T07:51:55.191795+0000","last_unstale":"2024-04-27T07:51:56.158129+0000","last_undegraded":"2024-04-27T07:51:56.158129+0000","last_fullsized":"2024-04-27T07:51:56.158129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:28.176405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155744+0000","last_change":"2024-04-27T07:51:55.174727+0000","last_active":"2024-04-27T07:51:56.155744+0000","last_peered":"2024-04-27T07:51:56.155744+0000","last_clean":"2024-04-27T07:51:56.155744+0000","last_became_active":"2024-04-27T07:51:55.174292+0000","last_became_peered":"2024-04-27T07:51:55.174292+0000","last_unstale":"2024-04-27T07:51:56.155744+0000","last_undegraded":"2024-04-27T07:51:56.155744+0000","last_fullsized":"2024-04-27T07:51:56.155744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:23:44.174069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358921+0000","last_change":"2024-04-27T07:51:55.168229+0000","last_active":"2024-04-27T07:51:56.358921+0000","last_peered":"2024-04-27T07:51:56.358921+0000","last_clean":"2024-04-27T07:51:56.358921+0000","last_became_active":"2024-04-27T07:51:55.167503+0000","last_became_peered":"2024-04-27T07:51:55.167503+0000","last_unstale":"2024-04-27T07:51:56.358921+0000","last_undegraded":"2024-04-27T07:51:56.358921+0000","last_fullsized":"2024-04-27T07:51:56.358921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:41:31.842640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163568+0000","last_change":"2024-04-27T07:51:55.175811+0000","last_active":"2024-04-27T07:51:56.163568+0000","last_peered":"2024-04-27T07:51:56.163568+0000","last_clean":"2024-04-27T07:51:56.163568+0000","last_became_active":"2024-04-27T07:51:55.175525+0000","last_became_peered":"2024-04-27T07:51:55.175525+0000","last_unstale":"2024-04-27T07:51:56.163568+0000","last_undegraded":"2024-04-27T07:51:56.163568+0000","last_fullsized":"2024-04-27T07:51:56.163568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:23:52.641179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358534+0000","last_change":"2024-04-27T07:51:55.176629+0000","last_active":"2024-04-27T07:51:56.358534+0000","last_peered":"2024-04-27T07:51:56.358534+0000","last_clean":"2024-04-27T07:51:56.358534+0000","last_became_active":"2024-04-27T07:51:55.176339+0000","last_became_peered":"2024-04-27T07:51:55.176339+0000","last_unstale":"2024-04-27T07:51:56.358534+0000","last_undegraded":"2024-04-27T07:51:56.358534+0000","last_fullsized":"2024-04-27T07:51:56.358534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:34.753412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163796+0000","last_change":"2024-04-27T07:51:55.197584+0000","last_active":"2024-04-27T07:51:56.163796+0000","last_peered":"2024-04-27T07:51:56.163796+0000","last_clean":"2024-04-27T07:51:56.163796+0000","last_became_active":"2024-04-27T07:51:55.197251+0000","last_became_peered":"2024-04-27T07:51:55.197251+0000","last_unstale":"2024-04-27T07:51:56.163796+0000","last_undegraded":"2024-04-27T07:51:56.163796+0000","last_fullsized":"2024-04-27T07:51:56.163796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:03:27.316042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360945+0000","last_change":"2024-04-27T07:51:55.170496+0000","last_active":"2024-04-27T07:51:56.360945+0000","last_peered":"2024-04-27T07:51:56.360945+0000","last_clean":"2024-04-27T07:51:56.360945+0000","last_became_active":"2024-04-27T07:51:55.169457+0000","last_became_peered":"2024-04-27T07:51:55.169457+0000","last_unstale":"2024-04-27T07:51:56.360945+0000","last_undegraded":"2024-04-27T07:51:56.360945+0000","last_fullsized":"2024-04-27T07:51:56.360945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:16.089427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158632+0000","last_change":"2024-04-27T07:51:55.191102+0000","last_active":"2024-04-27T07:51:56.158632+0000","last_peered":"2024-04-27T07:51:56.158632+0000","last_clean":"2024-04-27T07:51:56.158632+0000","last_became_active":"2024-04-27T07:51:55.190216+0000","last_became_peered":"2024-04-27T07:51:55.190216+0000","last_unstale":"2024-04-27T07:51:56.158632+0000","last_undegraded":"2024-04-27T07:51:56.158632+0000","last_fullsized":"2024-04-27T07:51:56.158632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:15.797290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359208+0000","last_change":"2024-04-27T07:51:55.200839+0000","last_active":"2024-04-27T07:51:56.359208+0000","last_peered":"2024-04-27T07:51:56.359208+0000","last_clean":"2024-04-27T07:51:56.359208+0000","last_became_active":"2024-04-27T07:51:55.193370+0000","last_became_peered":"2024-04-27T07:51:55.193370+0000","last_unstale":"2024-04-27T07:51:56.359208+0000","last_undegraded":"2024-04-27T07:51:56.359208+0000","last_fullsized":"2024-04-27T07:51:56.359208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:55.155855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360151+0000","last_change":"2024-04-27T07:51:55.169522+0000","last_active":"2024-04-27T07:51:56.360151+0000","last_peered":"2024-04-27T07:51:56.360151+0000","last_clean":"2024-04-27T07:51:56.360151+0000","last_became_active":"2024-04-27T07:51:55.166086+0000","last_became_peered":"2024-04-27T07:51:55.166086+0000","last_unstale":"2024-04-27T07:51:56.360151+0000","last_undegraded":"2024-04-27T07:51:56.360151+0000","last_fullsized":"2024-04-27T07:51:56.360151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:50:23.363287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358951+0000","last_change":"2024-04-27T07:51:55.206346+0000","last_active":"2024-04-27T07:51:56.358951+0000","last_peered":"2024-04-27T07:51:56.358951+0000","last_clean":"2024-04-27T07:51:56.358951+0000","last_became_active":"2024-04-27T07:51:55.205742+0000","last_became_peered":"2024-04-27T07:51:55.205742+0000","last_unstale":"2024-04-27T07:51:56.358951+0000","last_undegraded":"2024-04-27T07:51:56.358951+0000","last_fullsized":"2024-04-27T07:51:56.358951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:32:10.146181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156740+0000","last_change":"2024-04-27T07:51:55.200655+0000","last_active":"2024-04-27T07:51:56.156740+0000","last_peered":"2024-04-27T07:51:56.156740+0000","last_clean":"2024-04-27T07:51:56.156740+0000","last_became_active":"2024-04-27T07:51:55.200348+0000","last_became_peered":"2024-04-27T07:51:55.200348+0000","last_unstale":"2024-04-27T07:51:56.156740+0000","last_undegraded":"2024-04-27T07:51:56.156740+0000","last_fullsized":"2024-04-27T07:51:56.156740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:14.466571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361232+0000","last_change":"2024-04-27T07:51:54.137725+0000","last_active":"2024-04-27T07:51:56.361232+0000","last_peered":"2024-04-27T07:51:56.361232+0000","last_clean":"2024-04-27T07:51:56.361232+0000","last_became_active":"2024-04-27T07:51:54.136080+0000","last_became_peered":"2024-04-27T07:51:54.136080+0000","last_unstale":"2024-04-27T07:51:56.361232+0000","last_undegraded":"2024-04-27T07:51:56.361232+0000","last_fullsized":"2024-04-27T07:51:56.361232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:03.819220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361243+0000","last_change":"2024-04-27T07:51:55.194268+0000","last_active":"2024-04-27T07:51:56.361243+0000","last_peered":"2024-04-27T07:51:56.361243+0000","last_clean":"2024-04-27T07:51:56.361243+0000","last_became_active":"2024-04-27T07:51:55.193615+0000","last_became_peered":"2024-04-27T07:51:55.193615+0000","last_unstale":"2024-04-27T07:51:56.361243+0000","last_undegraded":"2024-04-27T07:51:56.361243+0000","last_fullsized":"2024-04-27T07:51:56.361243+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:18:09.850466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156661+0000","last_change":"2024-04-27T07:51:54.122462+0000","last_active":"2024-04-27T07:51:56.156661+0000","last_peered":"2024-04-27T07:51:56.156661+0000","last_clean":"2024-04-27T07:51:56.156661+0000","last_became_active":"2024-04-27T07:51:54.121723+0000","last_became_peered":"2024-04-27T07:51:54.121723+0000","last_unstale":"2024-04-27T07:51:56.156661+0000","last_undegraded":"2024-04-27T07:51:56.156661+0000","last_fullsized":"2024-04-27T07:51:56.156661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:44:55.866398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159007+0000","last_change":"2024-04-27T07:51:55.197429+0000","last_active":"2024-04-27T07:51:56.159007+0000","last_peered":"2024-04-27T07:51:56.159007+0000","last_clean":"2024-04-27T07:51:56.159007+0000","last_became_active":"2024-04-27T07:51:55.191819+0000","last_became_peered":"2024-04-27T07:51:55.191819+0000","last_unstale":"2024-04-27T07:51:56.159007+0000","last_undegraded":"2024-04-27T07:51:56.159007+0000","last_fullsized":"2024-04-27T07:51:56.159007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:31.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357835+0000","last_change":"2024-04-27T07:51:54.113993+0000","last_active":"2024-04-27T07:51:56.357835+0000","last_peered":"2024-04-27T07:51:56.357835+0000","last_clean":"2024-04-27T07:51:56.357835+0000","last_became_active":"2024-04-27T07:51:54.112772+0000","last_became_peered":"2024-04-27T07:51:54.112772+0000","last_unstale":"2024-04-27T07:51:56.357835+0000","last_undegraded":"2024-04-27T07:51:56.357835+0000","last_fullsized":"2024-04-27T07:51:56.357835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:14.788660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163969+0000","last_change":"2024-04-27T07:51:55.191318+0000","last_active":"2024-04-27T07:51:56.163969+0000","last_peered":"2024-04-27T07:51:56.163969+0000","last_clean":"2024-04-27T07:51:56.163969+0000","last_became_active":"2024-04-27T07:51:55.190978+0000","last_became_peered":"2024-04-27T07:51:55.190978+0000","last_unstale":"2024-04-27T07:51:56.163969+0000","last_undegraded":"2024-04-27T07:51:56.163969+0000","last_fullsized":"2024-04-27T07:51:56.163969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:08:40.482291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154839+0000","last_change":"2024-04-27T07:51:54.117646+0000","last_active":"2024-04-27T07:51:56.154839+0000","last_peered":"2024-04-27T07:51:56.154839+0000","last_clean":"2024-04-27T07:51:56.154839+0000","last_became_active":"2024-04-27T07:51:54.117130+0000","last_became_peered":"2024-04-27T07:51:54.117130+0000","last_unstale":"2024-04-27T07:51:56.154839+0000","last_undegraded":"2024-04-27T07:51:56.154839+0000","last_fullsized":"2024-04-27T07:51:56.154839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:31.067921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359616+0000","last_change":"2024-04-27T07:51:55.202425+0000","last_active":"2024-04-27T07:51:56.359616+0000","last_peered":"2024-04-27T07:51:56.359616+0000","last_clean":"2024-04-27T07:51:56.359616+0000","last_became_active":"2024-04-27T07:51:55.201723+0000","last_became_peered":"2024-04-27T07:51:55.201723+0000","last_unstale":"2024-04-27T07:51:56.359616+0000","last_undegraded":"2024-04-27T07:51:56.359616+0000","last_fullsized":"2024-04-27T07:51:56.359616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:10:08.361934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158919+0000","last_change":"2024-04-27T07:51:54.138877+0000","last_active":"2024-04-27T07:51:56.158919+0000","last_peered":"2024-04-27T07:51:56.158919+0000","last_clean":"2024-04-27T07:51:56.158919+0000","last_became_active":"2024-04-27T07:51:54.137251+0000","last_became_peered":"2024-04-27T07:51:54.137251+0000","last_unstale":"2024-04-27T07:51:56.158919+0000","last_undegraded":"2024-04-27T07:51:56.158919+0000","last_fullsized":"2024-04-27T07:51:56.158919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:03.861737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360914+0000","last_change":"2024-04-27T07:51:55.170525+0000","last_active":"2024-04-27T07:51:56.360914+0000","last_peered":"2024-04-27T07:51:56.360914+0000","last_clean":"2024-04-27T07:51:56.360914+0000","last_became_active":"2024-04-27T07:51:55.169768+0000","last_became_peered":"2024-04-27T07:51:55.169768+0000","last_unstale":"2024-04-27T07:51:56.360914+0000","last_undegraded":"2024-04-27T07:51:56.360914+0000","last_fullsized":"2024-04-27T07:51:56.360914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:34.465556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158393+0000","last_change":"2024-04-27T07:51:54.130222+0000","last_active":"2024-04-27T07:51:56.158393+0000","last_peered":"2024-04-27T07:51:56.158393+0000","last_clean":"2024-04-27T07:51:56.158393+0000","last_became_active":"2024-04-27T07:51:54.129483+0000","last_became_peered":"2024-04-27T07:51:54.129483+0000","last_unstale":"2024-04-27T07:51:56.158393+0000","last_undegraded":"2024-04-27T07:51:56.158393+0000","last_fullsized":"2024-04-27T07:51:56.158393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157955+0000","last_change":"2024-04-27T07:51:55.199308+0000","last_active":"2024-04-27T07:51:56.157955+0000","last_peered":"2024-04-27T07:51:56.157955+0000","last_clean":"2024-04-27T07:51:56.157955+0000","last_became_active":"2024-04-27T07:51:55.198740+0000","last_became_peered":"2024-04-27T07:51:55.198740+0000","last_unstale":"2024-04-27T07:51:56.157955+0000","last_undegraded":"2024-04-27T07:51:56.157955+0000","last_fullsized":"2024-04-27T07:51:56.157955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:57.706999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357367+0000","last_change":"2024-04-27T07:51:54.119289+0000","last_active":"2024-04-27T07:51:56.357367+0000","last_peered":"2024-04-27T07:51:56.357367+0000","last_clean":"2024-04-27T07:51:56.357367+0000","last_became_active":"2024-04-27T07:51:54.118743+0000","last_became_peered":"2024-04-27T07:51:54.118743+0000","last_unstale":"2024-04-27T07:51:56.357367+0000","last_undegraded":"2024-04-27T07:51:56.357367+0000","last_fullsized":"2024-04-27T07:51:56.357367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:56.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163213+0000","last_change":"2024-04-27T07:51:55.184764+0000","last_active":"2024-04-27T07:51:56.163213+0000","last_peered":"2024-04-27T07:51:56.163213+0000","last_clean":"2024-04-27T07:51:56.163213+0000","last_became_active":"2024-04-27T07:51:55.184530+0000","last_became_peered":"2024-04-27T07:51:55.184530+0000","last_unstale":"2024-04-27T07:51:56.163213+0000","last_undegraded":"2024-04-27T07:51:56.163213+0000","last_fullsized":"2024-04-27T07:51:56.163213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:56:15.797290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157766+0000","last_change":"2024-04-27T07:51:54.143742+0000","last_active":"2024-04-27T07:51:56.157766+0000","last_peered":"2024-04-27T07:51:56.157766+0000","last_clean":"2024-04-27T07:51:56.157766+0000","last_became_active":"2024-04-27T07:51:54.138744+0000","last_became_peered":"2024-04-27T07:51:54.138744+0000","last_unstale":"2024-04-27T07:51:56.157766+0000","last_undegraded":"2024-04-27T07:51:56.157766+0000","last_fullsized":"2024-04-27T07:51:56.157766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:44.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360560+0000","last_change":"2024-04-27T07:51:55.175898+0000","last_active":"2024-04-27T07:51:56.360560+0000","last_peered":"2024-04-27T07:51:56.360560+0000","last_clean":"2024-04-27T07:51:56.360560+0000","last_became_active":"2024-04-27T07:51:55.175061+0000","last_became_peered":"2024-04-27T07:51:55.175061+0000","last_unstale":"2024-04-27T07:51:56.360560+0000","last_undegraded":"2024-04-27T07:51:56.360560+0000","last_fullsized":"2024-04-27T07:51:56.360560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:12:16.083170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358682+0000","last_change":"2024-04-27T07:51:54.132734+0000","last_active":"2024-04-27T07:51:56.358682+0000","last_peered":"2024-04-27T07:51:56.358682+0000","last_clean":"2024-04-27T07:51:56.358682+0000","last_became_active":"2024-04-27T07:51:54.132374+0000","last_became_peered":"2024-04-27T07:51:54.132374+0000","last_unstale":"2024-04-27T07:51:56.358682+0000","last_undegraded":"2024-04-27T07:51:56.358682+0000","last_fullsized":"2024-04-27T07:51:56.358682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:47.730389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358236+0000","last_change":"2024-04-27T07:51:55.166684+0000","last_active":"2024-04-27T07:51:56.358236+0000","last_peered":"2024-04-27T07:51:56.358236+0000","last_clean":"2024-04-27T07:51:56.358236+0000","last_became_active":"2024-04-27T07:51:55.162281+0000","last_became_peered":"2024-04-27T07:51:55.162281+0000","last_unstale":"2024-04-27T07:51:56.358236+0000","last_undegraded":"2024-04-27T07:51:56.358236+0000","last_fullsized":"2024-04-27T07:51:56.358236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:29.864584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156758+0000","last_change":"2024-04-27T07:51:54.119029+0000","last_active":"2024-04-27T07:51:56.156758+0000","last_peered":"2024-04-27T07:51:56.156758+0000","last_clean":"2024-04-27T07:51:56.156758+0000","last_became_active":"2024-04-27T07:51:54.118506+0000","last_became_peered":"2024-04-27T07:51:54.118506+0000","last_unstale":"2024-04-27T07:51:56.156758+0000","last_undegraded":"2024-04-27T07:51:56.156758+0000","last_fullsized":"2024-04-27T07:51:56.156758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.856412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358165+0000","last_change":"2024-04-27T07:51:55.179381+0000","last_active":"2024-04-27T07:51:56.358165+0000","last_peered":"2024-04-27T07:51:56.358165+0000","last_clean":"2024-04-27T07:51:56.358165+0000","last_became_active":"2024-04-27T07:51:55.176583+0000","last_became_peered":"2024-04-27T07:51:55.176583+0000","last_unstale":"2024-04-27T07:51:56.358165+0000","last_undegraded":"2024-04-27T07:51:56.358165+0000","last_fullsized":"2024-04-27T07:51:56.358165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:51:19.108138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.362040+0000","last_change":"2024-04-27T07:51:54.134902+0000","last_active":"2024-04-27T07:51:56.362040+0000","last_peered":"2024-04-27T07:51:56.362040+0000","last_clean":"2024-04-27T07:51:56.362040+0000","last_became_active":"2024-04-27T07:51:54.134080+0000","last_became_peered":"2024-04-27T07:51:54.134080+0000","last_unstale":"2024-04-27T07:51:56.362040+0000","last_undegraded":"2024-04-27T07:51:56.362040+0000","last_fullsized":"2024-04-27T07:51:56.362040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:27.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361820+0000","last_change":"2024-04-27T07:51:55.166422+0000","last_active":"2024-04-27T07:51:56.361820+0000","last_peered":"2024-04-27T07:51:56.361820+0000","last_clean":"2024-04-27T07:51:56.361820+0000","last_became_active":"2024-04-27T07:51:55.164684+0000","last_became_peered":"2024-04-27T07:51:55.164684+0000","last_unstale":"2024-04-27T07:51:56.361820+0000","last_undegraded":"2024-04-27T07:51:56.361820+0000","last_fullsized":"2024-04-27T07:51:56.361820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:51:19.108138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358436+0000","last_change":"2024-04-27T07:51:54.111787+0000","last_active":"2024-04-27T07:51:56.358436+0000","last_peered":"2024-04-27T07:51:56.358436+0000","last_clean":"2024-04-27T07:51:56.358436+0000","last_became_active":"2024-04-27T07:51:54.111117+0000","last_became_peered":"2024-04-27T07:51:54.111117+0000","last_unstale":"2024-04-27T07:51:56.358436+0000","last_undegraded":"2024-04-27T07:51:56.358436+0000","last_fullsized":"2024-04-27T07:51:56.358436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:40:27.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163081+0000","last_change":"2024-04-27T07:51:55.184665+0000","last_active":"2024-04-27T07:51:56.163081+0000","last_peered":"2024-04-27T07:51:56.163081+0000","last_clean":"2024-04-27T07:51:56.163081+0000","last_became_active":"2024-04-27T07:51:55.184403+0000","last_became_peered":"2024-04-27T07:51:55.184403+0000","last_unstale":"2024-04-27T07:51:56.163081+0000","last_undegraded":"2024-04-27T07:51:56.163081+0000","last_fullsized":"2024-04-27T07:51:56.163081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:22:48.739020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158952+0000","last_change":"2024-04-27T07:51:54.138567+0000","last_active":"2024-04-27T07:51:56.158952+0000","last_peered":"2024-04-27T07:51:56.158952+0000","last_clean":"2024-04-27T07:51:56.158952+0000","last_became_active":"2024-04-27T07:51:54.137816+0000","last_became_peered":"2024-04-27T07:51:54.137816+0000","last_unstale":"2024-04-27T07:51:56.158952+0000","last_undegraded":"2024-04-27T07:51:56.158952+0000","last_fullsized":"2024-04-27T07:51:56.158952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:01:57.470600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.358076+0000","last_change":"2024-04-27T07:51:55.171580+0000","last_active":"2024-04-27T07:51:56.358076+0000","last_peered":"2024-04-27T07:51:56.358076+0000","last_clean":"2024-04-27T07:51:56.358076+0000","last_became_active":"2024-04-27T07:51:55.168092+0000","last_became_peered":"2024-04-27T07:51:55.168092+0000","last_unstale":"2024-04-27T07:51:56.358076+0000","last_undegraded":"2024-04-27T07:51:56.358076+0000","last_fullsized":"2024-04-27T07:51:56.358076+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:16.089427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.162972+0000","last_change":"2024-04-27T07:51:54.125851+0000","last_active":"2024-04-27T07:51:56.162972+0000","last_peered":"2024-04-27T07:51:56.162972+0000","last_clean":"2024-04-27T07:51:56.162972+0000","last_became_active":"2024-04-27T07:51:54.125383+0000","last_became_peered":"2024-04-27T07:51:54.125383+0000","last_unstale":"2024-04-27T07:51:56.162972+0000","last_undegraded":"2024-04-27T07:51:56.162972+0000","last_fullsized":"2024-04-27T07:51:56.162972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.361318+0000","last_change":"2024-04-27T07:51:55.206748+0000","last_active":"2024-04-27T07:51:56.361318+0000","last_peered":"2024-04-27T07:51:56.361318+0000","last_clean":"2024-04-27T07:51:56.361318+0000","last_became_active":"2024-04-27T07:51:55.205761+0000","last_became_peered":"2024-04-27T07:51:55.205761+0000","last_unstale":"2024-04-27T07:51:56.361318+0000","last_undegraded":"2024-04-27T07:51:56.361318+0000","last_fullsized":"2024-04-27T07:51:56.361318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:12:19.821551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357190+0000","last_change":"2024-04-27T07:51:54.126199+0000","last_active":"2024-04-27T07:51:56.357190+0000","last_peered":"2024-04-27T07:51:56.357190+0000","last_clean":"2024-04-27T07:51:56.357190+0000","last_became_active":"2024-04-27T07:51:54.125879+0000","last_became_peered":"2024-04-27T07:51:54.125879+0000","last_unstale":"2024-04-27T07:51:56.357190+0000","last_undegraded":"2024-04-27T07:51:56.357190+0000","last_fullsized":"2024-04-27T07:51:56.357190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:23:43.165438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360525+0000","last_change":"2024-04-27T07:51:55.194912+0000","last_active":"2024-04-27T07:51:56.360525+0000","last_peered":"2024-04-27T07:51:56.360525+0000","last_clean":"2024-04-27T07:51:56.360525+0000","last_became_active":"2024-04-27T07:51:55.193885+0000","last_became_peered":"2024-04-27T07:51:55.193885+0000","last_unstale":"2024-04-27T07:51:56.360525+0000","last_undegraded":"2024-04-27T07:51:56.360525+0000","last_fullsized":"2024-04-27T07:51:56.360525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:40.907109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357181+0000","last_change":"2024-04-27T07:51:54.113113+0000","last_active":"2024-04-27T07:51:56.357181+0000","last_peered":"2024-04-27T07:51:56.357181+0000","last_clean":"2024-04-27T07:51:56.357181+0000","last_became_active":"2024-04-27T07:51:54.112816+0000","last_became_peered":"2024-04-27T07:51:54.112816+0000","last_unstale":"2024-04-27T07:51:56.357181+0000","last_undegraded":"2024-04-27T07:51:56.357181+0000","last_fullsized":"2024-04-27T07:51:56.357181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:51:26.633386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159296+0000","last_change":"2024-04-27T07:51:55.190533+0000","last_active":"2024-04-27T07:51:56.159296+0000","last_peered":"2024-04-27T07:51:56.159296+0000","last_clean":"2024-04-27T07:51:56.159296+0000","last_became_active":"2024-04-27T07:51:55.189962+0000","last_became_peered":"2024-04-27T07:51:55.189962+0000","last_unstale":"2024-04-27T07:51:56.159296+0000","last_undegraded":"2024-04-27T07:51:56.159296+0000","last_fullsized":"2024-04-27T07:51:56.159296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:30.207284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163680+0000","last_change":"2024-04-27T07:51:54.121271+0000","last_active":"2024-04-27T07:51:56.163680+0000","last_peered":"2024-04-27T07:51:56.163680+0000","last_clean":"2024-04-27T07:51:56.163680+0000","last_became_active":"2024-04-27T07:51:54.120834+0000","last_became_peered":"2024-04-27T07:51:54.120834+0000","last_unstale":"2024-04-27T07:51:56.163680+0000","last_undegraded":"2024-04-27T07:51:56.163680+0000","last_fullsized":"2024-04-27T07:51:56.163680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:08.726549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360699+0000","last_change":"2024-04-27T07:51:55.194049+0000","last_active":"2024-04-27T07:51:56.360699+0000","last_peered":"2024-04-27T07:51:56.360699+0000","last_clean":"2024-04-27T07:51:56.360699+0000","last_became_active":"2024-04-27T07:51:55.193139+0000","last_became_peered":"2024-04-27T07:51:55.193139+0000","last_unstale":"2024-04-27T07:51:56.360699+0000","last_undegraded":"2024-04-27T07:51:56.360699+0000","last_fullsized":"2024-04-27T07:51:56.360699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:51:36.231986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159252+0000","last_change":"2024-04-27T07:51:54.144802+0000","last_active":"2024-04-27T07:51:56.159252+0000","last_peered":"2024-04-27T07:51:56.159252+0000","last_clean":"2024-04-27T07:51:56.159252+0000","last_became_active":"2024-04-27T07:51:54.144063+0000","last_became_peered":"2024-04-27T07:51:54.144063+0000","last_unstale":"2024-04-27T07:51:56.159252+0000","last_undegraded":"2024-04-27T07:51:56.159252+0000","last_fullsized":"2024-04-27T07:51:56.159252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:25.687567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163495+0000","last_change":"2024-04-27T07:51:55.197786+0000","last_active":"2024-04-27T07:51:56.163495+0000","last_peered":"2024-04-27T07:51:56.163495+0000","last_clean":"2024-04-27T07:51:56.163495+0000","last_became_active":"2024-04-27T07:51:55.197271+0000","last_became_peered":"2024-04-27T07:51:55.197271+0000","last_unstale":"2024-04-27T07:51:56.163495+0000","last_undegraded":"2024-04-27T07:51:56.163495+0000","last_fullsized":"2024-04-27T07:51:56.163495+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:23:20.564940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.159284+0000","last_change":"2024-04-27T07:51:54.144467+0000","last_active":"2024-04-27T07:51:56.159284+0000","last_peered":"2024-04-27T07:51:56.159284+0000","last_clean":"2024-04-27T07:51:56.159284+0000","last_became_active":"2024-04-27T07:51:54.143451+0000","last_became_peered":"2024-04-27T07:51:54.143451+0000","last_unstale":"2024-04-27T07:51:56.159284+0000","last_undegraded":"2024-04-27T07:51:56.159284+0000","last_fullsized":"2024-04-27T07:51:56.159284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:00:05.467783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155125+0000","last_change":"2024-04-27T07:51:55.163330+0000","last_active":"2024-04-27T07:51:56.155125+0000","last_peered":"2024-04-27T07:51:56.155125+0000","last_clean":"2024-04-27T07:51:56.155125+0000","last_became_active":"2024-04-27T07:51:55.163003+0000","last_became_peered":"2024-04-27T07:51:55.163003+0000","last_unstale":"2024-04-27T07:51:56.155125+0000","last_undegraded":"2024-04-27T07:51:56.155125+0000","last_fullsized":"2024-04-27T07:51:56.155125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:16:47.348161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158988+0000","last_change":"2024-04-27T07:51:54.139192+0000","last_active":"2024-04-27T07:51:56.158988+0000","last_peered":"2024-04-27T07:51:56.158988+0000","last_clean":"2024-04-27T07:51:56.158988+0000","last_became_active":"2024-04-27T07:51:54.137084+0000","last_became_peered":"2024-04-27T07:51:54.137084+0000","last_unstale":"2024-04-27T07:51:56.158988+0000","last_undegraded":"2024-04-27T07:51:56.158988+0000","last_fullsized":"2024-04-27T07:51:56.158988+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:17:11.463660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154952+0000","last_change":"2024-04-27T07:51:55.191255+0000","last_active":"2024-04-27T07:51:56.154952+0000","last_peered":"2024-04-27T07:51:56.154952+0000","last_clean":"2024-04-27T07:51:56.154952+0000","last_became_active":"2024-04-27T07:51:55.190911+0000","last_became_peered":"2024-04-27T07:51:55.190911+0000","last_unstale":"2024-04-27T07:51:56.154952+0000","last_undegraded":"2024-04-27T07:51:56.154952+0000","last_fullsized":"2024-04-27T07:51:56.154952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:48:31.253337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156464+0000","last_change":"2024-04-27T07:51:54.117841+0000","last_active":"2024-04-27T07:51:56.156464+0000","last_peered":"2024-04-27T07:51:56.156464+0000","last_clean":"2024-04-27T07:51:56.156464+0000","last_became_active":"2024-04-27T07:51:54.117462+0000","last_became_peered":"2024-04-27T07:51:54.117462+0000","last_unstale":"2024-04-27T07:51:56.156464+0000","last_undegraded":"2024-04-27T07:51:56.156464+0000","last_fullsized":"2024-04-27T07:51:56.156464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:50.409799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.155202+0000","last_change":"2024-04-27T07:51:55.166521+0000","last_active":"2024-04-27T07:51:56.155202+0000","last_peered":"2024-04-27T07:51:56.155202+0000","last_clean":"2024-04-27T07:51:56.155202+0000","last_became_active":"2024-04-27T07:51:55.166043+0000","last_became_peered":"2024-04-27T07:51:55.166043+0000","last_unstale":"2024-04-27T07:51:56.155202+0000","last_undegraded":"2024-04-27T07:51:56.155202+0000","last_fullsized":"2024-04-27T07:51:56.155202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:54.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359351+0000","last_change":"2024-04-27T07:51:54.132661+0000","last_active":"2024-04-27T07:51:56.359351+0000","last_peered":"2024-04-27T07:51:56.359351+0000","last_clean":"2024-04-27T07:51:56.359351+0000","last_became_active":"2024-04-27T07:51:54.131852+0000","last_became_peered":"2024-04-27T07:51:54.131852+0000","last_unstale":"2024-04-27T07:51:56.359351+0000","last_undegraded":"2024-04-27T07:51:56.359351+0000","last_fullsized":"2024-04-27T07:51:56.359351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:21.051122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154514+0000","last_change":"2024-04-27T07:51:55.164715+0000","last_active":"2024-04-27T07:51:56.154514+0000","last_peered":"2024-04-27T07:51:56.154514+0000","last_clean":"2024-04-27T07:51:56.154514+0000","last_became_active":"2024-04-27T07:51:55.164419+0000","last_became_peered":"2024-04-27T07:51:55.164419+0000","last_unstale":"2024-04-27T07:51:56.154514+0000","last_undegraded":"2024-04-27T07:51:56.154514+0000","last_fullsized":"2024-04-27T07:51:56.154514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:38.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.157994+0000","last_change":"2024-04-27T07:51:54.136947+0000","last_active":"2024-04-27T07:51:56.157994+0000","last_peered":"2024-04-27T07:51:56.157994+0000","last_clean":"2024-04-27T07:51:56.157994+0000","last_became_active":"2024-04-27T07:51:54.131725+0000","last_became_peered":"2024-04-27T07:51:54.131725+0000","last_unstale":"2024-04-27T07:51:56.157994+0000","last_undegraded":"2024-04-27T07:51:56.157994+0000","last_fullsized":"2024-04-27T07:51:56.157994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:20:32.938716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359219+0000","last_change":"2024-04-27T07:51:55.187793+0000","last_active":"2024-04-27T07:51:56.359219+0000","last_peered":"2024-04-27T07:51:56.359219+0000","last_clean":"2024-04-27T07:51:56.359219+0000","last_became_active":"2024-04-27T07:51:55.185683+0000","last_became_peered":"2024-04-27T07:51:55.185683+0000","last_unstale":"2024-04-27T07:51:56.359219+0000","last_undegraded":"2024-04-27T07:51:56.359219+0000","last_fullsized":"2024-04-27T07:51:56.359219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:33:02.075078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359570+0000","last_change":"2024-04-27T07:51:54.134411+0000","last_active":"2024-04-27T07:51:56.359570+0000","last_peered":"2024-04-27T07:51:56.359570+0000","last_clean":"2024-04-27T07:51:56.359570+0000","last_became_active":"2024-04-27T07:51:54.133697+0000","last_became_peered":"2024-04-27T07:51:54.133697+0000","last_unstale":"2024-04-27T07:51:56.359570+0000","last_undegraded":"2024-04-27T07:51:56.359570+0000","last_fullsized":"2024-04-27T07:51:56.359570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:13.457940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.154799+0000","last_change":"2024-04-27T07:51:55.172934+0000","last_active":"2024-04-27T07:51:56.154799+0000","last_peered":"2024-04-27T07:51:56.154799+0000","last_clean":"2024-04-27T07:51:56.154799+0000","last_became_active":"2024-04-27T07:51:55.172600+0000","last_became_peered":"2024-04-27T07:51:55.172600+0000","last_unstale":"2024-04-27T07:51:56.154799+0000","last_undegraded":"2024-04-27T07:51:56.154799+0000","last_fullsized":"2024-04-27T07:51:56.154799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:09.735179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163760+0000","last_change":"2024-04-27T07:51:54.122872+0000","last_active":"2024-04-27T07:51:56.163760+0000","last_peered":"2024-04-27T07:51:56.163760+0000","last_clean":"2024-04-27T07:51:56.163760+0000","last_became_active":"2024-04-27T07:51:54.122391+0000","last_became_peered":"2024-04-27T07:51:54.122391+0000","last_unstale":"2024-04-27T07:51:56.163760+0000","last_undegraded":"2024-04-27T07:51:56.163760+0000","last_fullsized":"2024-04-27T07:51:56.163760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:53:13.052684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360823+0000","last_change":"2024-04-27T07:51:55.164723+0000","last_active":"2024-04-27T07:51:56.360823+0000","last_peered":"2024-04-27T07:51:56.360823+0000","last_clean":"2024-04-27T07:51:56.360823+0000","last_became_active":"2024-04-27T07:51:55.163877+0000","last_became_peered":"2024-04-27T07:51:55.163877+0000","last_unstale":"2024-04-27T07:51:56.360823+0000","last_undegraded":"2024-04-27T07:51:56.360823+0000","last_fullsized":"2024-04-27T07:51:56.360823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:32.692411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.161937+0000","last_change":"2024-04-27T07:51:54.112256+0000","last_active":"2024-04-27T07:51:56.161937+0000","last_peered":"2024-04-27T07:51:56.161937+0000","last_clean":"2024-04-27T07:51:56.161937+0000","last_became_active":"2024-04-27T07:51:54.111768+0000","last_became_peered":"2024-04-27T07:51:54.111768+0000","last_unstale":"2024-04-27T07:51:56.161937+0000","last_undegraded":"2024-04-27T07:51:56.161937+0000","last_fullsized":"2024-04-27T07:51:56.161937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:37.723837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156691+0000","last_change":"2024-04-27T07:51:55.163718+0000","last_active":"2024-04-27T07:51:56.156691+0000","last_peered":"2024-04-27T07:51:56.156691+0000","last_clean":"2024-04-27T07:51:56.156691+0000","last_became_active":"2024-04-27T07:51:55.163432+0000","last_became_peered":"2024-04-27T07:51:55.163432+0000","last_unstale":"2024-04-27T07:51:56.156691+0000","last_undegraded":"2024-04-27T07:51:56.156691+0000","last_fullsized":"2024-04-27T07:51:56.156691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:57.706999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163543+0000","last_change":"2024-04-27T07:51:54.108254+0000","last_active":"2024-04-27T07:51:56.163543+0000","last_peered":"2024-04-27T07:51:56.163543+0000","last_clean":"2024-04-27T07:51:56.163543+0000","last_became_active":"2024-04-27T07:51:54.107946+0000","last_became_peered":"2024-04-27T07:51:54.107946+0000","last_unstale":"2024-04-27T07:51:56.163543+0000","last_undegraded":"2024-04-27T07:51:56.163543+0000","last_fullsized":"2024-04-27T07:51:56.163543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:45:50.409799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359367+0000","last_change":"2024-04-27T07:51:55.181437+0000","last_active":"2024-04-27T07:51:56.359367+0000","last_peered":"2024-04-27T07:51:56.359367+0000","last_clean":"2024-04-27T07:51:56.359367+0000","last_became_active":"2024-04-27T07:51:55.180517+0000","last_became_peered":"2024-04-27T07:51:55.180517+0000","last_unstale":"2024-04-27T07:51:56.359367+0000","last_undegraded":"2024-04-27T07:51:56.359367+0000","last_fullsized":"2024-04-27T07:51:56.359367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:32:07.108034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156679+0000","last_change":"2024-04-27T07:51:54.120139+0000","last_active":"2024-04-27T07:51:56.156679+0000","last_peered":"2024-04-27T07:51:56.156679+0000","last_clean":"2024-04-27T07:51:56.156679+0000","last_became_active":"2024-04-27T07:51:54.119538+0000","last_became_peered":"2024-04-27T07:51:54.119538+0000","last_unstale":"2024-04-27T07:51:56.156679+0000","last_undegraded":"2024-04-27T07:51:56.156679+0000","last_fullsized":"2024-04-27T07:51:56.156679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:08.726549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.156542+0000","last_change":"2024-04-27T07:51:55.169350+0000","last_active":"2024-04-27T07:51:56.156542+0000","last_peered":"2024-04-27T07:51:56.156542+0000","last_clean":"2024-04-27T07:51:56.156542+0000","last_became_active":"2024-04-27T07:51:55.169006+0000","last_became_peered":"2024-04-27T07:51:55.169006+0000","last_unstale":"2024-04-27T07:51:56.156542+0000","last_undegraded":"2024-04-27T07:51:56.156542+0000","last_fullsized":"2024-04-27T07:51:56.156542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:51:27.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.163057+0000","last_change":"2024-04-27T07:51:54.125639+0000","last_active":"2024-04-27T07:51:56.163057+0000","last_peered":"2024-04-27T07:51:56.163057+0000","last_clean":"2024-04-27T07:51:56.163057+0000","last_became_active":"2024-04-27T07:51:54.125374+0000","last_became_peered":"2024-04-27T07:51:54.125374+0000","last_unstale":"2024-04-27T07:51:56.163057+0000","last_undegraded":"2024-04-27T07:51:56.163057+0000","last_fullsized":"2024-04-27T07:51:56.163057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.856412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357201+0000","last_change":"2024-04-27T07:51:55.157964+0000","last_active":"2024-04-27T07:51:56.357201+0000","last_peered":"2024-04-27T07:51:56.357201+0000","last_clean":"2024-04-27T07:51:56.357201+0000","last_became_active":"2024-04-27T07:51:55.157519+0000","last_became_peered":"2024-04-27T07:51:55.157519+0000","last_unstale":"2024-04-27T07:51:56.357201+0000","last_undegraded":"2024-04-27T07:51:56.357201+0000","last_fullsized":"2024-04-27T07:51:56.357201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:12:16.083170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158133+0000","last_change":"2024-04-27T07:51:54.144370+0000","last_active":"2024-04-27T07:51:56.158133+0000","last_peered":"2024-04-27T07:51:56.158133+0000","last_clean":"2024-04-27T07:51:56.158133+0000","last_became_active":"2024-04-27T07:51:54.143612+0000","last_became_peered":"2024-04-27T07:51:54.143612+0000","last_unstale":"2024-04-27T07:51:56.158133+0000","last_undegraded":"2024-04-27T07:51:56.158133+0000","last_fullsized":"2024-04-27T07:51:56.158133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:44:55.866398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.158084+0000","last_change":"2024-04-27T07:51:55.192563+0000","last_active":"2024-04-27T07:51:56.158084+0000","last_peered":"2024-04-27T07:51:56.158084+0000","last_clean":"2024-04-27T07:51:56.158084+0000","last_became_active":"2024-04-27T07:51:55.191806+0000","last_became_peered":"2024-04-27T07:51:55.191806+0000","last_unstale":"2024-04-27T07:51:56.158084+0000","last_undegraded":"2024-04-27T07:51:56.158084+0000","last_fullsized":"2024-04-27T07:51:56.158084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:14.466571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.357462+0000","last_change":"2024-04-27T07:51:54.132729+0000","last_active":"2024-04-27T07:51:56.357462+0000","last_peered":"2024-04-27T07:51:56.357462+0000","last_clean":"2024-04-27T07:51:56.357462+0000","last_became_active":"2024-04-27T07:51:54.132309+0000","last_became_peered":"2024-04-27T07:51:54.132309+0000","last_unstale":"2024-04-27T07:51:56.357462+0000","last_undegraded":"2024-04-27T07:51:56.357462+0000","last_fullsized":"2024-04-27T07:51:56.357462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:13.457940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360155+0000","last_change":"2024-04-27T07:51:55.186929+0000","last_active":"2024-04-27T07:51:56.360155+0000","last_peered":"2024-04-27T07:51:56.360155+0000","last_clean":"2024-04-27T07:51:56.360155+0000","last_became_active":"2024-04-27T07:51:55.176976+0000","last_became_peered":"2024-04-27T07:51:55.176976+0000","last_unstale":"2024-04-27T07:51:56.360155+0000","last_undegraded":"2024-04-27T07:51:56.360155+0000","last_fullsized":"2024-04-27T07:51:56.360155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:54.089885+0000","last_clean_scrub_stamp":"2024-04-27T07:51:54.089885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:29:07.528389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.360821+0000","last_change":"2024-04-27T07:51:54.146133+0000","last_active":"2024-04-27T07:51:56.360821+0000","last_peered":"2024-04-27T07:51:56.360821+0000","last_clean":"2024-04-27T07:51:56.360821+0000","last_became_active":"2024-04-27T07:51:54.145807+0000","last_became_peered":"2024-04-27T07:51:54.145807+0000","last_unstale":"2024-04-27T07:51:56.360821+0000","last_undegraded":"2024-04-27T07:51:56.360821+0000","last_fullsized":"2024-04-27T07:51:56.360821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:51:53.081255+0000","last_clean_scrub_stamp":"2024-04-27T07:51:53.081255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:51:32.021029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-27T07:51:56.359505+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:51:56.359505+0000","last_peered":"2024-04-27T07:51:56.359505+0000","last_clean":"2024-04-27T07:51:56.359505+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:51:56.359505+0000","last_undegraded":"2024-04-27T07:51:56.359505+0000","last_fullsized":"2024-04-27T07:51:56.359505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":740295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":149941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":151471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":385024,"data_stored":149907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":151471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":740205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:51:59.589 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-27T07:51:59.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-27T07:51:59.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.754+0000 7f278bcb7640 1 Processor -- start 2024-04-27T07:51:59.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.754+0000 7f278bcb7640 1 -- start start 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f27840b2a70 0x7f278415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840cbfb0 con 0x7f27840cd050 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840cc290 con 0x7f27840cd470 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f27840cc570 con 0x7f27840b2a70 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f27840b2a70 0x7f278415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f27840b2a70 0x7f278415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37898/0 (socket says 172.21.15.22:37898) 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 -- 172.21.15.22:0/3767877882 learned_addr learned my addr 172.21.15.22:0/3767877882 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 -- 172.21.15.22:0/3767877882 >> v1:172.21.15.183:6789/0 conn(0x7f27840cd470 legacy=0x7f27840b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 -- 172.21.15.22:0/3767877882 >> v1:172.21.15.22:6789/0 conn(0x7f27840cd050 legacy=0x7f278415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 -- 172.21.15.22:0/3767877882 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27840cc850 con 0x7f27840b2a70 2024-04-27T07:51:59.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2789a53640 1 --2- 172.21.15.22:0/3767877882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840b2a70 0x7f278415f460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2780009870 tx=0x7f278001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f49de811850bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2788a51640 1 -- 172.21.15.22:0/3767877882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800b0070 con 0x7f27840b2a70 2024-04-27T07:51:59.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2788a51640 1 -- 172.21.15.22:0/3767877882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27800ab400 con 0x7f27840b2a70 2024-04-27T07:51:59.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f2788a51640 1 -- 172.21.15.22:0/3767877882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800ab7d0 con 0x7f27840b2a70 2024-04-27T07:51:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 -- 172.21.15.22:0/3767877882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840b2a70 msgr2=0x7f278415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.758+0000 7f278bcb7640 1 --2- 172.21.15.22:0/3767877882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840b2a70 0x7f278415f460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2780009870 tx=0x7f278001c490 comp rx=0 tx=0).stop 2024-04-27T07:51:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/3767877882 shutdown_connections 2024-04-27T07:51:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 --2- 172.21.15.22:0/3767877882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840b2a70 0x7f278415f460 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/3767877882 >> 172.21.15.22:0/3767877882 conn(0x7f278409aa30 msgr2=0x7f27840b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:51:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/3767877882 shutdown_connections 2024-04-27T07:51:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/3767877882 wait complete. 2024-04-27T07:51:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 Processor -- start 2024-04-27T07:51:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- start start 2024-04-27T07:51:59.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27840b2a70 0x7f27840d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27840cd050 0x7f27840c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2784169930 con 0x7f27840cd470 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27841697b0 con 0x7f27840cd050 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2784169ab0 con 0x7f27840b2a70 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37914/0 (socket says 172.21.15.22:37914) 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 -- 172.21.15.22:0/996638939 learned_addr learned my addr 172.21.15.22:0/996638939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f2789a53640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27840b2a70 0x7f27840d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f2789252640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27840cd050 0x7f27840c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 -- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27840b2a70 msgr2=0x7f27840d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27840b2a70 0x7f27840d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 -- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27840cd050 msgr2=0x7f27840c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27840cd050 0x7f27840c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 -- 172.21.15.22:0/996638939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2784164e60 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278a254640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b9c0 tx=0x7f277c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c019070 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c00cba0 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841f7a30 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c014440 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841f76e0 con 0x7f27840cd470 2024-04-27T07:51:59.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.762+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278404ff10 con 0x7f27840cd470 2024-04-27T07:51:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.766+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f277c01c070 con 0x7f27840cd470 2024-04-27T07:51:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.766+0000 7f277a7fc640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 0x7f276806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:51:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.766+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f277c086660 con 0x7f27840cd470 2024-04-27T07:51:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.766+0000 7f2789a53640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 0x7f276806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:51:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.766+0000 7f2789a53640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 0x7f276806a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27800ad000 tx=0x7f27800a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:51:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.770+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f277c056cb0 con 0x7f27840cd470 2024-04-27T07:51:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.994+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f278409af10 con 0x7f27840cd470 2024-04-27T07:51:59.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.994+0000 7f277a7fc640 1 -- 172.21.15.22:0/996638939 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f277c0565b0 con 0x7f27840cd470 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAnrixmLWgVGhAAj3agIuftac+O2Tov9zpCZQ== 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-27T07:51:59.999 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 msgr2=0x7f276806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 0x7f276806a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27800ad000 tx=0x7f27800a6000 comp rx=0 tx=0).stop 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 msgr2=0x7f27840c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b9c0 tx=0x7f277c00be80 comp rx=0 tx=0).stop 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 shutdown_connections 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f2768068380 0x7f276806a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27840cd470 0x7f27840c6c00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27840cd050 0x7f27840c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 --2- 172.21.15.22:0/996638939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27840b2a70 0x7f27840d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 >> 172.21.15.22:0/996638939 conn(0x7f278409aa30 msgr2=0x7f278406f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 shutdown_connections 2024-04-27T07:52:00.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:51:59.998+0000 7f278bcb7640 1 -- 172.21.15.22:0/996638939 wait complete. 2024-04-27T07:52:00.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:52:00.080 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-04-27T07:52:00.078+0000 7f5c0f429640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:52:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 Processor -- start 2024-04-27T07:52:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 -- start start 2024-04-27T07:52:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe1540b0910 0x7fe1540b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1540b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1540afe20 con 0x7fe15415b620 2024-04-27T07:52:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe1540b4830 con 0x7fe1540b0910 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe159e0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe1540af580 con 0x7fe1540d1bd0 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe152ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe1540b0910 0x7fe1540b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe1537fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe15415b620 0x7fe1540d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44062/0 (socket says 172.21.15.22:44062) 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe1537fe640 1 -- 172.21.15.22:0/2344553344 learned_addr learned my addr 172.21.15.22:0/2344553344 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe153fff640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1540b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe152ffd640 1 -- 172.21.15.22:0/2344553344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 msgr2=0x7fe1540b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe152ffd640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1540b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe152ffd640 1 -- 172.21.15.22:0/2344553344 >> v1:172.21.15.22:6789/0 conn(0x7fe15415b620 legacy=0x7fe1540d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe1527fc640 1 -- 172.21.15.22:0/2344553344 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910671000 0 0) 0x7fe1540afe20 con 0x7fe15415b620 2024-04-27T07:52:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.142+0000 7fe152ffd640 1 -- 172.21.15.22:0/2344553344 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1540af860 con 0x7fe1540b0910 2024-04-27T07:52:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe152ffd640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe1540b0d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097c0 tx=0x7fe13c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e19f67e5ca552c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe1527fc640 1 -- 172.21.15.22:0/2344553344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2070 con 0x7fe1540b0910 2024-04-27T07:52:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe1527fc640 1 -- 172.21.15.22:0/2344553344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c0ad640 con 0x7fe1540b0910 2024-04-27T07:52:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe1527fc640 1 -- 172.21.15.22:0/2344553344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0ada10 con 0x7fe1540b0910 2024-04-27T07:52:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- 172.21.15.22:0/2344553344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 msgr2=0x7fe1540b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe1540b0d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097c0 tx=0x7fe13c01c4a0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- 172.21.15.22:0/2344553344 shutdown_connections 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1540b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 --2- 172.21.15.22:0/2344553344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe1540b0d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- 172.21.15.22:0/2344553344 >> 172.21.15.22:0/2344553344 conn(0x7fe15409aa30 msgr2=0x7fe1540b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- 172.21.15.22:0/2344553344 shutdown_connections 2024-04-27T07:52:00.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- 172.21.15.22:0/2344553344 wait complete. 2024-04-27T07:52:00.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 Processor -- start 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 -- start start 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.146+0000 7fe159e0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1541526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe15415b620 0x7fe1541f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540b5650 con 0x7fe1540b0910 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540b54d0 con 0x7fe15415b620 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe1540b57d0 con 0x7fe1540d1bd0 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60032/0 (socket says 172.21.15.22:60032) 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 -- 172.21.15.22:0/827536966 learned_addr learned my addr 172.21.15.22:0/827536966 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe152ffd640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1541526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe153fff640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe15415b620 0x7fe1541f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 -- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 msgr2=0x7fe1541526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1541526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 -- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe15415b620 msgr2=0x7fe1541f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe15415b620 0x7fe1541f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 -- 172.21.15.22:0/827536966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe154154a10 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe152ffd640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1541526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe1537fe640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c480 tx=0x7fe14400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144018070 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe144007500 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144004a30 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe154165da0 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1540ce920 con 0x7fe1540b0910 2024-04-27T07:52:00.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154069340 con 0x7fe1540b0910 2024-04-27T07:52:00.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe14401b070 con 0x7fe1540b0910 2024-04-27T07:52:00.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 0x7fe13406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe144085c60 con 0x7fe1540b0910 2024-04-27T07:52:00.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.150+0000 7fe152ffd640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 0x7fe13406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.154+0000 7fe152ffd640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 0x7fe13406a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0af000 tx=0x7fe13c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.154+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe144056290 con 0x7fe1540b0910 2024-04-27T07:52:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe154151d60 con 0x7fe1540b0910 2024-04-27T07:52:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe14bfff640 1 -- 172.21.15.22:0/827536966 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fe144055b90 con 0x7fe1540b0910 2024-04-27T07:52:00.357 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:52:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:52:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:52:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 msgr2=0x7fe13406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 0x7fe13406a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0af000 tx=0x7fe13c0a7070 comp rx=0 tx=0).stop 2024-04-27T07:52:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 msgr2=0x7fe15414a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c480 tx=0x7fe14400c940 comp rx=0 tx=0).stop 2024-04-27T07:52:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 shutdown_connections 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fe134068380 0x7fe13406a840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe15415b620 0x7fe1541f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe1540d1bd0 0x7fe1541526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 --2- 172.21.15.22:0/827536966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1540b0910 0x7fe15414a610 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 >> 172.21.15.22:0/827536966 conn(0x7fe15409aa30 msgr2=0x7fe15415a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 shutdown_connections 2024-04-27T07:52:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.354+0000 7fe159e0b640 1 -- 172.21.15.22:0/827536966 wait complete. 2024-04-27T07:52:00.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-27T07:51:54.480954+0000', 'modified': '2024-04-27T07:51:57.571057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5229': {'gid': 5229, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6834/3341477719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3341477719}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3341477719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5229, 'qdb_cluster': [5229]} 2024-04-27T07:52:00.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-27T07:52:00.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-27T07:52:00.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 Processor -- start 2024-04-27T07:52:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 -- start start 2024-04-27T07:52:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c1607e0 0x7f7b4c15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 0x7f7b4c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c160c40 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c15cba0 con 0x7f7b4c15c710 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.906+0000 7f7b50b84640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c0cd1e0 con 0x7f7b4c1607e0 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60036/0 (socket says 172.21.15.22:60036) 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 -- 172.21.15.22:0/954320002 learned_addr learned my addr 172.21.15.22:0/954320002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c1607e0 0x7f7b4c15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/954320002 >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 0x7f7b4c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 -- 172.21.15.22:0/954320002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c1607e0 msgr2=0x7f7b4c15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c1607e0 0x7f7b4c15c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 -- 172.21.15.22:0/954320002 >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 msgr2=0x7f7b4c0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/954320002 >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 0x7f7b4c0b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 -- 172.21.15.22:0/954320002 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c0cd4c0 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/954320002 >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 0x7f7b4c0b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:52:00.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0ccb60 tx=0x7f7b3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d66fd92d38b4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4959a640 1 -- 172.21.15.22:0/954320002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ab620 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4959a640 1 -- 172.21.15.22:0/954320002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b340b0070 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b4959a640 1 -- 172.21.15.22:0/954320002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ab9e0 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 -- 172.21.15.22:0/954320002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c0b1db0 msgr2=0x7f7b4c1602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0ccb60 tx=0x7f7b3401c460 comp rx=0 tx=0).stop 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 -- 172.21.15.22:0/954320002 shutdown_connections 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 --2- 172.21.15.22:0/954320002 >> v2:172.21.15.183:3300/0 conn(0x7f7b4c15c710 0x7f7b4c0b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c1607e0 0x7f7b4c15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 --2- 172.21.15.22:0/954320002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c0b1db0 0x7f7b4c1602a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 -- 172.21.15.22:0/954320002 >> 172.21.15.22:0/954320002 conn(0x7f7b4c0d2580 msgr2=0x7f7b4c0d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 -- 172.21.15.22:0/954320002 shutdown_connections 2024-04-27T07:52:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.910+0000 7f7b50b84640 1 -- 172.21.15.22:0/954320002 wait complete. 2024-04-27T07:52:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 Processor -- start 2024-04-27T07:52:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- start start 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c0b1db0 0x7f7b4c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 0x7f7b4c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b3d10 con 0x7f7b4c15c710 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b3b90 con 0x7f7b4c1607e0 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c09b240 con 0x7f7b4c0b1db0 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60038/0 (socket says 172.21.15.22:60038) 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 -- 172.21.15.22:0/21260616 learned_addr learned my addr 172.21.15.22:0/21260616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c0b1db0 0x7f7b4c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 0x7f7b4c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 -- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c0b1db0 msgr2=0x7f7b4c0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c0b1db0 0x7f7b4c0c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 -- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 msgr2=0x7f7b4c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 0x7f7b4c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 -- 172.21.15.22:0/21260616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c148f20 con 0x7f7b4c15c710 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 0x7f7b4c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d620 tx=0x7f7b3c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b42ffd640 1 -- 172.21.15.22:0/21260616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00b620 con 0x7f7b4c15c710 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b42ffd640 1 -- 172.21.15.22:0/21260616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3c019070 con 0x7f7b4c15c710 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c15d090 con 0x7f7b4c15c710 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b42ffd640 1 -- 172.21.15.22:0/21260616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00b9e0 con 0x7f7b4c15c710 2024-04-27T07:52:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.914+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c159f80 con 0x7f7b4c15c710 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b42ffd640 1 -- 172.21.15.22:0/21260616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7b3c00bb80 con 0x7f7b4c15c710 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b42ffd640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 0x7f7b2c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b42ffd640 1 -- 172.21.15.22:0/21260616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7b3c086e90 con 0x7f7b4c15c710 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 0x7f7b2c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 Processor -- start 2024-04-27T07:52:00.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 -- start start 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 0x7f7b2c06a870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b340af000 tx=0x7f7b340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561da50f3be0 con 0x561da52a2fd0 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561da508dbf0 con 0x561da52effb0 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x561da50efba0 con 0x561da52ef690 2024-04-27T07:52:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60044/0 (socket says 172.21.15.22:60044) 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 -- 172.21.15.22:0/221473789 learned_addr learned my addr 172.21.15.22:0/221473789 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 -- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 msgr2=0x561da52efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 -- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 msgr2=0x561da52a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 -- 172.21.15.22:0/221473789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da519b630 con 0x561da52a2fd0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d4e0 tx=0x7f7b4400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a31738a00a5e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b2bfff640 1 -- 172.21.15.22:0/221473789 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4401a070 con 0x561da52a2fd0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b2bfff640 1 -- 172.21.15.22:0/221473789 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4400b760 con 0x561da52a2fd0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.918+0000 7f7b2bfff640 1 -- 172.21.15.22:0/221473789 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400bb10 con 0x561da52a2fd0 2024-04-27T07:52:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/221473789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 msgr2=0x561da52a33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d4e0 tx=0x7f7b4400d9a0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/221473789 shutdown_connections 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52a33b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- 172.21.15.22:0/221473789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/221473789 >> 172.21.15.22:0/221473789 conn(0x561da529ca70 msgr2=0x561da52d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/221473789 shutdown_connections 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/221473789 wait complete. 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- start start 2024-04-27T07:52:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 Processor -- start 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561da51afd10 con 0x561da52a2fd0 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561da509d3a0 con 0x561da52effb0 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x561da50b71c0 con 0x561da52ef690 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 10 client.0 _ll_register_callbacks cb 0x561da5290730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60046/0 (socket says 172.21.15.22:60046) 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1921250348 learned_addr learned my addr 172.21.15.22:0/1921250348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4ad9d640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 msgr2=0x561da52f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 msgr2=0x561da52d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1921250348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da51390d0 con 0x561da52a2fd0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ad040 tx=0x7f7b3401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fff839fc038b7ca server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340cc060 con 0x561da52a2fd0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7b340be600 con 0x561da52a2fd0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b4400b760 con 0x561da52a2fd0 2024-04-27T07:52:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f7b340d2050 con 0x561da52a2fd0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 10 client.5412 fetch_fsmap learned FSMap version 16 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x561da5079e80 con 0x561da52a2fd0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4400b760 con 0x561da52a2fd0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f7b340d5020 con 0x561da52a2fd0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 10 client.5412 fetch_fsmap finished waiting for FSMap version 16 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 10 client.5412 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 20 client.5412 populate_metadata read hostname 'smithi022' 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52d43b0 0x561da52d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 4 client.5412 mds_command: new command op to 5229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 --> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da508cbc0 con 0x561da52d43b0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52d43b0 0x561da52d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b297fa640 10 client.5412 ms_handle_connect on v2:172.21.15.22:6832/3341477719 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.922+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52d43b0 0x561da52d4790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e6e1497c0706f8 server_cookie=c070a04f429a19b8 in_seq=0 out_seq=0 2024-04-27T07:52:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b297fa640 1 -- 172.21.15.22:0/1921250348 <== mds.0 v2:172.21.15.22:6832/3341477719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561da508cbc0 con 0x561da52d43b0 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b297fa640 10 client.5412 handle_command_reply: tid=0 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 client.5412 shutdown 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 msgr2=0x561da52f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ad040 tx=0x7f7b3401aca0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 shutdown_connections 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52d43b0 0x561da52d4790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da52f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1921250348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52f69f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b297fa640 0 client.5412 ms_handle_reset on v2:172.21.15.22:6832/3341477719 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 >> 172.21.15.22:0/1921250348 conn(0x561da529ca70 msgr2=0x561da52a2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 shutdown_connections 2024-04-27T07:52:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 1 -- 172.21.15.22:0/1921250348 wait complete. 2024-04-27T07:52:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.926+0000 7f7b53d05000 20 client.5412 trim_cache size 0 max 16384 2024-04-27T07:52:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 Processor -- start 2024-04-27T07:52:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 -- start start 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52ef690 0x561da52a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52effb0 0x561da52a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c15f710 con 0x561da52a2fd0 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c15ea70 con 0x561da52ef690 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c15f0c0 con 0x561da52effb0 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60058/0 (socket says 172.21.15.22:60058) 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 -- 172.21.15.22:0/3568702273 learned_addr learned my addr 172.21.15.22:0/3568702273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 -- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52effb0 msgr2=0x561da52a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52effb0 0x561da52a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 -- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52ef690 msgr2=0x561da52a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52ef690 0x561da52a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.942+0000 7f7b4a59c640 1 -- 172.21.15.22:0/3568702273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da52a26d0 con 0x561da52a2fd0 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b3401cd00 tx=0x7f7b340c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cf7808d669c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b2b7fe640 1 -- 172.21.15.22:0/3568702273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ba070 con 0x561da52a2fd0 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b2b7fe640 1 -- 172.21.15.22:0/3568702273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b340b0040 con 0x561da52a2fd0 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b2b7fe640 1 -- 172.21.15.22:0/3568702273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ab4b0 con 0x561da52a2fd0 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/3568702273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 msgr2=0x561da52ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b3401cd00 tx=0x7f7b340c3000 comp rx=0 tx=0).stop 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/3568702273 shutdown_connections 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52effb0 0x561da52a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52ef690 0x561da52a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- 172.21.15.22:0/3568702273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da52ab220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/3568702273 >> 172.21.15.22:0/3568702273 conn(0x561da529ca70 msgr2=0x561da52fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/3568702273 shutdown_connections 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/3568702273 wait complete. 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- start start 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 Processor -- start 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da5363730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561da50f2ef0 con 0x561da52a2fd0 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561da50fed00 con 0x561da52effb0 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x561da50f8210 con 0x561da52ef690 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 10 client.0 _ll_register_callbacks cb 0x7f7b340096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:52:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60072/0 (socket says 172.21.15.22:60072) 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1588133063 learned_addr learned my addr 172.21.15.22:0/1588133063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 msgr2=0x561da5363730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da5363730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 msgr2=0x561da52f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 -- 172.21.15.22:0/1588133063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da52efde0 con 0x561da52a2fd0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b4a59c640 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009630 tx=0x7f7b34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce2c6aec53a0f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ba040 con 0x561da52a2fd0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7b34006040 con 0x561da52a2fd0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b340b0040 con 0x561da52a2fd0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f7b34006280 con 0x561da52a2fd0 2024-04-27T07:52:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 10 client.5418 fetch_fsmap learned FSMap version 16 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x561da50bbb30 con 0x561da52a2fd0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b340b0040 con 0x561da52a2fd0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f7b340d5040 con 0x561da52a2fd0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 10 client.5418 fetch_fsmap finished waiting for FSMap version 16 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 10 client.5418 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 20 client.5418 populate_metadata read hostname 'smithi022' 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52f3db0 0x561da52f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 4 client.5418 mds_command: new command op to 5229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 --> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] -- command(tid 0: {"prefix": "status"}) -- 0x561da51017f0 con 0x561da52f3db0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.946+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52f3db0 0x561da52f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b49d9b640 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52f3db0 0x561da52f4190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4995c610e9d0d11 server_cookie=78a4f45348b71ea2 in_seq=0 out_seq=0 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b297fa640 10 client.5418 ms_handle_connect on v2:172.21.15.22:6832/3341477719 2024-04-27T07:52:00.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b297fa640 1 -- 172.21.15.22:0/1588133063 <== mds.0 v2:172.21.15.22:6832/3341477719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561da51017f0 con 0x561da52f3db0 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b297fa640 10 client.5418 handle_command_reply: tid=0 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 client.5418 shutdown 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 msgr2=0x561da53631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009630 tx=0x7f7b34008040 comp rx=0 tx=0).stop 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 shutdown_connections 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:6832/3341477719,v1:172.21.15.22:6834/3341477719] conn(0x561da52f3db0 0x561da52f4190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561da52effb0 0x561da52f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x561da52ef690 0x561da5363730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b297fa640 0 client.5418 ms_handle_reset on v2:172.21.15.22:6832/3341477719 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 --2- 172.21.15.22:0/1588133063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561da52a2fd0 0x561da53631f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 >> 172.21.15.22:0/1588133063 conn(0x561da529ca70 msgr2=0x561da52a73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 shutdown_connections 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 1 -- 172.21.15.22:0/1588133063 wait complete. 2024-04-27T07:52:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b53d05000 20 client.5418 trim_cache size 0 max 16384 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "807b2bc3-0b23-4239-8346-8619bcda9f4e", 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5229, 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.7962197030000002, 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 16, 2024-04-27T07:52:00.959 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 34, 2024-04-27T07:52:00.960 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 34, 2024-04-27T07:52:00.960 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 12.745192754 2024-04-27T07:52:00.960 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:52:00.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 msgr2=0x7f7b2c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 0x7f7b2c06a870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b340af000 tx=0x7f7b340a7000 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 msgr2=0x7f7b4c1489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d620 tx=0x7f7b3c00dae0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 shutdown_connections 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f7b2c0683b0 0x7f7b2c06a870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b4c1607e0 0x7f7b4c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b4c15c710 0x7f7b4c1489e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 --2- 172.21.15.22:0/21260616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b4c0b1db0 0x7f7b4c0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 >> 172.21.15.22:0/21260616 conn(0x7f7b4c0d2580 msgr2=0x7f7b4c0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 shutdown_connections 2024-04-27T07:52:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:00.950+0000 7f7b50b84640 1 -- 172.21.15.22:0/21260616 wait complete. 2024-04-27T07:52:00.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:52:01.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 Processor -- start 2024-04-27T07:52:01.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 -- start start 2024-04-27T07:52:01.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1cc0144dd0 0x7f1cc006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc00720d0 con 0x7f1cc006f6a0 2024-04-27T07:52:01.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc0072370 con 0x7f1cc00cfe10 2024-04-27T07:52:01.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cc5890640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc006fb00 con 0x7f1cc0144dd0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbe7fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1cc006f6a0 0x7f1cc00cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44070/0 (socket says 172.21.15.22:44070) 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbe7fc640 1 -- 172.21.15.22:0/1405416586 learned_addr learned my addr 172.21.15.22:0/1405416586 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1405416586 >> v2:172.21.15.183:3300/0 conn(0x7f1cc0144dd0 0x7f1cc006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027730867 0 0) 0x7f1cc00720d0 con 0x7f1cc006f6a0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cac003530 con 0x7f1cc006f6a0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3089406450 0 0) 0x7f1cac003530 con 0x7f1cc006f6a0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 >> v2:172.21.15.183:3300/0 conn(0x7f1cc0144dd0 msgr2=0x7f1cc006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 --2- 172.21.15.22:0/1405416586 >> v2:172.21.15.183:3300/0 conn(0x7f1cc0144dd0 0x7f1cc006f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 >> v1:172.21.15.183:6789/0 conn(0x7f1cc00cfe10 legacy=0x7f1cc00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0072650 con 0x7f1cc006f6a0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f1ca80016a0 con 0x7f1cc006f6a0 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 >> v1:172.21.15.22:6789/0 conn(0x7f1cc006f6a0 legacy=0x7f1cc00cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac004810 0x7f1cac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0051d0 0x7f1cac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac00c040 con 0x7f1cac0039c0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac00c1f0 con 0x7f1cac0051d0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac00c3f0 con 0x7f1cac004810 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbf7fe640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0051d0 0x7f1cac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbe7fc640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac004810 0x7f1cac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac004810 msgr2=0x7f1cac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac004810 0x7f1cac004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0051d0 msgr2=0x7f1cac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0051d0 0x7f1cac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1405416586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac003530 con 0x7f1cac0039c0 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbf7fe640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0051d0 0x7f1cac00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:52:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.074+0000 7f1cbf7fe640 1 -- 172.21.15.22:0/1405416586 reap_dead start 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cac003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009fc0 tx=0x7f1cb001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a414d7f618d8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b2070 con 0x7f1cac0039c0 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb00a5a40 con 0x7f1cac0039c0 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbdffb640 1 -- 172.21.15.22:0/1405416586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0009c10 con 0x7f1cac0039c0 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 msgr2=0x7f1cac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cac003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009fc0 tx=0x7f1cb001cc50 comp rx=0 tx=0).stop 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- 172.21.15.22:0/1405416586 shutdown_connections 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1405416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cac003de0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- 172.21.15.22:0/1405416586 >> 172.21.15.22:0/1405416586 conn(0x7f1cc0067f10 msgr2=0x7f1cc006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- 172.21.15.22:0/1405416586 shutdown_connections 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- 172.21.15.22:0/1405416586 wait complete. 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 Processor -- start 2024-04-27T07:52:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- start start 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 0x7f1cc0141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 0x7f1cc00ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00cfb10 con 0x7f1cac0039c0 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00cfc90 con 0x7f1cc00ca100 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cc5890640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc015c300 con 0x7f1cc0141a90 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60098/0 (socket says 172.21.15.22:60098) 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1035786620 learned_addr learned my addr 172.21.15.22:0/1035786620 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbf7fe640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 0x7f1cc00ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbe7fc640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 0x7f1cc0141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 msgr2=0x7f1cc0141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 0x7f1cc0141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 msgr2=0x7f1cc00ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 0x7f1cc00ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbeffd640 1 -- 172.21.15.22:0/1035786620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00cd370 con 0x7f1cac0039c0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbf7fe640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 0x7f1cc00ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.078+0000 7f1cbe7fc640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 0x7f1cc0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cbeffd640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1cb001c6c0 tx=0x7f1cb0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00ba070 con 0x7f1cac0039c0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb00adac0 con 0x7f1cac0039c0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc0142450 con 0x7f1cac0039c0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b2040 con 0x7f1cac0039c0 2024-04-27T07:52:01.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc00cc5b0 con 0x7f1cac0039c0 2024-04-27T07:52:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0068c30 con 0x7f1cac0039c0 2024-04-27T07:52:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1cb00b6410 con 0x7f1cac0039c0 2024-04-27T07:52:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 0x7f1ca006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb0121050 con 0x7f1cac0039c0 2024-04-27T07:52:01.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cbe7fc640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 0x7f1ca006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:01.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.082+0000 7f1cbe7fc640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 0x7f1ca006a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00b5220 tx=0x7f1ca80014a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:01.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.086+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1cc00cd370 con 0x7f1cac0039c0 2024-04-27T07:52:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.282+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1cc015fa50 con 0x7f1cac0039c0 2024-04-27T07:52:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cb7fff640 1 -- 172.21.15.22:0/1035786620 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f1cb0016990 con 0x7f1cac0039c0 2024-04-27T07:52:01.288 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 msgr2=0x7f1ca006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 0x7f1ca006a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00b5220 tx=0x7f1ca80014a0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 msgr2=0x7f1cc015d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1cb001c6c0 tx=0x7f1cb0009750 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 shutdown_connections 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ca00683b0 0x7f1ca006a870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cc00ca100 0x7f1cc00ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc0141a90 0x7f1cc0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 --2- 172.21.15.22:0/1035786620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0039c0 0x7f1cc015d7e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 >> 172.21.15.22:0/1035786620 conn(0x7f1cc0067f10 msgr2=0x7f1cc0143d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 shutdown_connections 2024-04-27T07:52:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01.286+0000 7f1cc5890640 1 -- 172.21.15.22:0/1035786620 wait complete. 2024-04-27T07:52:01.304 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:52:01.305 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:52:01.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.305 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-04-27T07:52:01.311 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:52:01.311 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:52:01.311 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-04-27T07:52:01.312 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:ba:2b:46:d0 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-27T07:52:01.313 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:52:01.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.315 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:52:01.365 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-04-27T07:52:01.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.366 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-27T07:52:01.413 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:52:01.413 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:52:01.413 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-27T07:52:01.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-27T07:52:01.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-27T07:52:01.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-27T07:52:01.414 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-27T07:52:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:52:01.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:52:01.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.537 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-27T07:52:01.543 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-27T07:52:01.543 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T07:52:01.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:52:01.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-04-27T07:52:01.601 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-27T07:52:01.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:52:01.601 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:01.648 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:52:01.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:52:01.649 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:01.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:52:01.753 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:52:01.801 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:52:01.802 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:01.803 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:01.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.804 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:52:01.852 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T07:52:01.852 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-27T07:52:01.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:52:01.912 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:52:01.960 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:52:01.961 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:01.962 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:01.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:01.963 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:52:01.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-04-27T07:52:01.974+0000 7f9620a29580 -1 init, newargv = 0x55ab61b53ba0 newargc=15 2024-04-27T07:52:01.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[36960]: starting ceph client 2024-04-27T07:52:01.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[36960]: starting fuse 2024-04-27T07:52:03.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-04-27T07:52:03.101 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:52:03.153 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-04-27T07:52:03.153 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-04-27T07:52:03.153 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-04-27T07:52:03.154 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:03.155 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:52:03.156 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:03.156 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:52:03.156 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:03.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.157 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-04-27T07:52:03.204 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-04-27T07:52:03.205 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-27T07:52:03.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:52:03.205 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-04-27T07:52:03.205 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-04-27T07:52:03.205 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-04-27T07:52:03.205 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:52:03.206 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-04-27T07:52:03.207 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-04-27T07:52:03.337 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.36960.asok 2024-04-27T07:52:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:52:03.350 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36960.asok 2024-04-27T07:52:03.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36960.asok status 2024-04-27T07:52:03.511 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:52:03.511 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-04-27T07:52:03.511 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:52:03.511 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "36960", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5430, 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 5430 2024-04-27T07:52:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3910834154 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3910834154 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-27T07:52:03.513 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.5430 192.168.0.1:0/3910834154", 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/3910834154", 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 16, 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 34, 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-04-27T07:52:03.514 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:52:03.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.526 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-04-27T07:52:03.526 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-04-27T07:52:03.533 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:03.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:52:03.588 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=xfstests-dev 2024-04-27T07:52:03.636 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.BsSpkJ5lk2xfstests-dev 2024-04-27T07:52:03.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.636 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.BsSpkJ5lk2xfstests-dev) 2024-04-27T07:52:03.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:03.697 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.BsSpkJ5lk2xfstests-dev) 2024-04-27T07:52:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:Cloning into '/tmp/tmp.BsSpkJ5lk2xfstests-dev'... 2024-04-27T07:52:04.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:04.585 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-27T07:52:04.642 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=test 2024-04-27T07:52:04.688 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.ywDIeOSfVMtest 2024-04-27T07:52:04.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:52:04.688 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-27T07:52:04.742 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=scratch 2024-04-27T07:52:04.792 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.ZBGWbtclM6scratch 2024-04-27T07:52:04.792 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "22.04". 2024-04-27T07:52:04.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool libtool-bin liburing-dev libuuid1 lvm2 make psmisc python3 quota sed uuid-dev uuid-runtime xfsprogs sqlite3 exfatprogs f2fs-tools ocfs2-tools udftools xfsdump xfslibs-dev 2024-04-27T07:52:04.904 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-04-27T07:52:05.114 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-04-27T07:52:05.115 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:acl is already the newest version (2.3.1-1). 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:acl set to manually installed. 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:attr is already the newest version (1:2.5.1-1build1). 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:automake is already the newest version (1:1.16.5-1.3). 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:bc is already the newest version (1.07.1-3build1). 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:gcc is already the newest version (4:11.2.0-1ubuntu1). 2024-04-27T07:52:05.319 INFO:teuthology.orchestra.run.smithi022.stdout:gcc set to manually installed. 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:libacl1-dev is already the newest version (2.3.1-1). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:libaio-dev is already the newest version (0.3.112-13build1). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:libtool is already the newest version (2.4.6-15build2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:lvm2 is already the newest version (2.03.11-2.1ubuntu4). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:make is already the newest version (4.3-4.1build1). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:make set to manually installed. 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:psmisc is already the newest version (23.4-2build3). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:quota is already the newest version (4.06-1build2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:sed is already the newest version (4.8-1ubuntu2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:xfsdump is already the newest version (3.1.9+0+nmu2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:52:05.320 INFO:teuthology.orchestra.run.smithi022.stdout:xfsprogs is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:dbench is already the newest version (4.0-2build1). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:dump is already the newest version (0.4b47-3). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:e2fsprogs is already the newest version (1.46.5-2ubuntu1.1). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:git is already the newest version (1:2.34.1-1ubuntu1.10). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:indent is already the newest version (2.2.12-1ubuntu0.22.04.1). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:libuuid1 is already the newest version (2.37.2-4ubuntu3.4). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:python3 is already the newest version (3.10.6-1~22.04). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3.4). 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2024-04-27T07:52:05.321 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2024-04-27T07:52:05.322 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2024-04-27T07:52:05.322 INFO:teuthology.orchestra.run.smithi022.stdout: python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:52:05.322 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:52:05.322 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-04-27T07:52:05.323 INFO:teuthology.orchestra.run.smithi022.stdout: libcap2 libcmap4 libcorosync-common4 libdlm3 libpmemblk1 libqb100 2024-04-27T07:52:05.323 INFO:teuthology.orchestra.run.smithi022.stdout: libsqlite3-0 2024-04-27T07:52:05.323 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-04-27T07:52:05.323 INFO:teuthology.orchestra.run.smithi022.stdout: gnuplot gfio python-scipy gawk-doc sqlite3-doc dvd+rw-tools udfclient 2024-04-27T07:52:05.356 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-04-27T07:52:05.356 INFO:teuthology.orchestra.run.smithi022.stdout: exfatprogs f2fs-tools fio libcap-dev libcmap4 libcorosync-common4 libdlm3 2024-04-27T07:52:05.357 INFO:teuthology.orchestra.run.smithi022.stdout: libgdbm-dev libpmemblk1 libqb100 liburing-dev ocfs2-tools sqlite3 udftools 2024-04-27T07:52:05.358 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2024-04-27T07:52:05.359 INFO:teuthology.orchestra.run.smithi022.stdout: gawk libcap2 libsqlite3-0 uuid-runtime 2024-04-27T07:52:05.441 INFO:teuthology.orchestra.run.smithi022.stdout:4 upgraded, 14 newly installed, 0 to remove and 294 not upgraded. 2024-04-27T07:52:05.441 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 7,314 kB of archives. 2024-04-27T07:52:05.441 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 14.2 MB of additional disk space will be used. 2024-04-27T07:52:05.441 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-runtime amd64 2.37.2-4ubuntu3.4 [32.0 kB] 2024-04-27T07:52:05.491 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gawk amd64 1:5.1.0-1ubuntu0.1 [447 kB] 2024-04-27T07:52:05.575 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcorosync-common4 amd64 3.1.6-1ubuntu1 [7,322 B] 2024-04-27T07:52:05.578 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqb100 amd64 2.0.4-1ubuntu0.2 [74.2 kB] 2024-04-27T07:52:05.582 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcmap4 amd64 3.1.6-1ubuntu1 [10.4 kB] 2024-04-27T07:52:05.584 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdlm3 amd64 4.1.1-1build1 [14.4 kB] 2024-04-27T07:52:05.586 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 ocfs2-tools amd64 1.8.7-1build1 [646 kB] 2024-04-27T07:52:05.610 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap2 amd64 1:2.44-1ubuntu0.22.04.1 [18.3 kB] 2024-04-27T07:52:05.617 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-27T07:52:05.635 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 exfatprogs amd64 1.1.3-1 [40.3 kB] 2024-04-27T07:52:05.638 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 f2fs-tools amd64 1.14.0-2build1 [169 kB] 2024-04-27T07:52:05.646 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2024-04-27T07:52:05.649 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2024-04-27T07:52:05.715 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap-dev amd64 1:2.44-1ubuntu0.22.04.1 [39.4 kB] 2024-04-27T07:52:05.716 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgdbm-dev amd64 1.23-1 [117 kB] 2024-04-27T07:52:05.717 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liburing-dev amd64 2.1-2build1 [51.8 kB] 2024-04-27T07:52:05.718 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-27T07:52:05.726 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 udftools amd64 2.3-1 [125 kB] 2024-04-27T07:52:06.042 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: unable to initialize frontend: Dialog 2024-04-27T07:52:06.042 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-27T07:52:06.042 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: falling back to frontend: Readline 2024-04-27T07:52:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: unable to initialize frontend: Readline 2024-04-27T07:52:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: (This frontend requires a controlling tty.) 2024-04-27T07:52:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: falling back to frontend: Teletype 2024-04-27T07:52:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-04-27T07:52:06.099 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 7,314 kB in 0s (20.3 MB/s) 2024-04-27T07:52:06.414 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176797 files and directories currently installed.) 2024-04-27T07:52:06.420 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-uuid-runtime_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:52:06.552 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking uuid-runtime (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:52:07.004 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-gawk_1%3a5.1.0-1ubuntu0.1_amd64.deb ... 2024-04-27T07:52:07.133 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking gawk (1:5.1.0-1ubuntu0.1) over (1:5.1.0-1build3) ... 2024-04-27T07:52:07.593 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcorosync-common4:amd64. 2024-04-27T07:52:07.622 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-libcorosync-common4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-27T07:52:07.693 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-27T07:52:08.172 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqb100:amd64. 2024-04-27T07:52:08.201 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-libqb100_2.0.4-1ubuntu0.2_amd64.deb ... 2024-04-27T07:52:08.238 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-27T07:52:08.575 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcmap4:amd64. 2024-04-27T07:52:08.604 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-libcmap4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-27T07:52:08.641 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-27T07:52:08.944 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libdlm3:amd64. 2024-04-27T07:52:08.973 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-libdlm3_4.1.1-1build1_amd64.deb ... 2024-04-27T07:52:09.011 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libdlm3:amd64 (4.1.1-1build1) ... 2024-04-27T07:52:09.357 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ocfs2-tools. 2024-04-27T07:52:09.385 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-ocfs2-tools_1.8.7-1build1_amd64.deb ... 2024-04-27T07:52:09.423 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ocfs2-tools (1.8.7-1build1) ... 2024-04-27T07:52:09.857 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-libcap2_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:52:10.003 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) over (1:2.44-1build3) ... 2024-04-27T07:52:10.341 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-27T07:52:10.855 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176871 files and directories currently installed.) 2024-04-27T07:52:10.861 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-27T07:52:10.995 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-27T07:52:11.307 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package exfatprogs. 2024-04-27T07:52:11.325 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-exfatprogs_1.1.3-1_amd64.deb ... 2024-04-27T07:52:11.357 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking exfatprogs (1.1.3-1) ... 2024-04-27T07:52:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package f2fs-tools. 2024-04-27T07:52:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-f2fs-tools_1.14.0-2build1_amd64.deb ... 2024-04-27T07:52:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking f2fs-tools (1.14.0-2build1) ... 2024-04-27T07:52:12.114 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpmemblk1:amd64. 2024-04-27T07:52:12.142 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-libpmemblk1_1.11.1-3build1_amd64.deb ... 2024-04-27T07:52:12.189 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-27T07:52:12.492 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package fio. 2024-04-27T07:52:12.521 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-fio_3.28-1_amd64.deb ... 2024-04-27T07:52:12.558 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking fio (3.28-1) ... 2024-04-27T07:52:13.012 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcap-dev:amd64. 2024-04-27T07:52:13.041 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-libcap-dev_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:52:13.079 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-27T07:52:13.342 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2024-04-27T07:52:13.371 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-libgdbm-dev_1.23-1_amd64.deb ... 2024-04-27T07:52:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libgdbm-dev:amd64 (1.23-1) ... 2024-04-27T07:52:13.712 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liburing-dev:amd64. 2024-04-27T07:52:13.729 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-liburing-dev_2.1-2build1_amd64.deb ... 2024-04-27T07:52:13.770 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liburing-dev:amd64 (2.1-2build1) ... 2024-04-27T07:52:14.032 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package sqlite3. 2024-04-27T07:52:14.049 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../8-sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-27T07:52:14.082 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-27T07:52:14.443 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package udftools. 2024-04-27T07:52:14.472 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../9-udftools_2.3-1_amd64.deb ... 2024-04-27T07:52:14.523 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking udftools (2.3-1) ... 2024-04-27T07:52:14.863 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up exfatprogs (1.1.3-1) ... 2024-04-27T07:52:14.990 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up gawk (1:5.1.0-1ubuntu0.1) ... 2024-04-27T07:52:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up f2fs-tools (1.14.0-2build1) ... 2024-04-27T07:52:15.343 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-27T07:52:15.452 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-27T07:52:15.578 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-27T07:52:15.704 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-27T07:52:15.839 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up uuid-runtime (2.37.2-4ubuntu3.4) ... 2024-04-27T07:52:16.725 INFO:teuthology.orchestra.run.smithi022.stdout:uuidd.service is a disabled or a static unit not running, not starting it. 2024-04-27T07:52:16.774 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up udftools (2.3-1) ... 2024-04-27T07:52:16.909 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libdlm3:amd64 (4.1.1-1build1) ... 2024-04-27T07:52:17.035 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libgdbm-dev:amd64 (1.23-1) ... 2024-04-27T07:52:17.161 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-27T07:52:17.287 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-27T07:52:17.413 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liburing-dev:amd64 (2.1-2build1) ... 2024-04-27T07:52:17.530 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-27T07:52:17.656 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up fio (3.28-1) ... 2024-04-27T07:52:18.175 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ocfs2-tools (1.8.7-1build1) ... 2024-04-27T07:52:18.431 INFO:teuthology.orchestra.run.smithi022.stdout:debconf: unable to initialize frontend: Dialog 2024-04-27T07:52:18.431 INFO:teuthology.orchestra.run.smithi022.stdout:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-27T07:52:18.431 INFO:teuthology.orchestra.run.smithi022.stdout:debconf: falling back to frontend: Readline 2024-04-27T07:52:19.220 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for initramfs-tools (0.140ubuntu13.1) ... 2024-04-27T07:52:19.327 INFO:teuthology.orchestra.run.smithi022.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-27T07:52:27.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:27.252 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:37.823 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:52:37.966 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:52:39.284 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:52:39.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd -m fsgqa 2024-04-27T07:52:39.755 INFO:teuthology.orchestra.run.smithi022.stderr:useradd: user 'fsgqa' already exists 2024-04-27T07:52:39.756 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-27T07:52:39.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo groupadd fsgqa 2024-04-27T07:52:39.808 INFO:teuthology.orchestra.run.smithi022.stderr:groupadd: group 'fsgqa' already exists 2024-04-27T07:52:39.809 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-27T07:52:39.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd fsgqa2 2024-04-27T07:52:40.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd 123456-fsgqa 2024-04-27T07:52:40.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-27T07:52:40.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 Processor -- start 2024-04-27T07:52:40.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 -- start start 2024-04-27T07:52:40.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa050145a00 0x7fa050072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:40.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05006e050 con 0x7fa0501454b0 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0500ceae0 con 0x7fa050072740 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa056607640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa0500cedc0 con 0x7fa050145a00 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa04ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa0501454b0 0x7fa050072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54920/0 (socket says 172.21.15.22:54920) 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa04ffff640 1 -- 172.21.15.22:0/2933189002 learned_addr learned my addr 172.21.15.22:0/2933189002 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa054ba4640 1 --2- 172.21.15.22:0/2933189002 >> v2:172.21.15.22:3300/0 conn(0x7fa050145a00 0x7fa050072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa04effd640 1 -- 172.21.15.22:0/2933189002 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086773571 0 0) 0x7fa05006e050 con 0x7fa0501454b0 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa04effd640 1 -- 172.21.15.22:0/2933189002 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa038003530 con 0x7fa0501454b0 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa054ba4640 1 -- 172.21.15.22:0/2933189002 >> v1:172.21.15.183:6789/0 conn(0x7fa050072740 legacy=0x7fa05015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa054ba4640 1 -- 172.21.15.22:0/2933189002 >> v1:172.21.15.22:6789/0 conn(0x7fa0501454b0 legacy=0x7fa050072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.633+0000 7fa054ba4640 1 -- 172.21.15.22:0/2933189002 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05006e310 con 0x7fa050145a00 2024-04-27T07:52:40.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa054ba4640 1 --2- 172.21.15.22:0/2933189002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050145a00 0x7fa050072fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa048009d90 tx=0x7fa04801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53c4fb1e03a389c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:40.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa04effd640 1 -- 172.21.15.22:0/2933189002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480b2070 con 0x7fa050145a00 2024-04-27T07:52:40.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa04effd640 1 -- 172.21.15.22:0/2933189002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0480a57f0 con 0x7fa050145a00 2024-04-27T07:52:40.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa04effd640 1 -- 172.21.15.22:0/2933189002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480a5bc0 con 0x7fa050145a00 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- 172.21.15.22:0/2933189002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050145a00 msgr2=0x7fa050072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 --2- 172.21.15.22:0/2933189002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050145a00 0x7fa050072fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa048009d90 tx=0x7fa04801c930 comp rx=0 tx=0).stop 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- 172.21.15.22:0/2933189002 shutdown_connections 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 --2- 172.21.15.22:0/2933189002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050145a00 0x7fa050072fa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- 172.21.15.22:0/2933189002 >> 172.21.15.22:0/2933189002 conn(0x7fa050067f10 msgr2=0x7fa05006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- 172.21.15.22:0/2933189002 shutdown_connections 2024-04-27T07:52:40.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- 172.21.15.22:0/2933189002 wait complete. 2024-04-27T07:52:40.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 Processor -- start 2024-04-27T07:52:40.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.637+0000 7fa056607640 1 -- start start 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 0x7fa05015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 0x7fa05015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa050145a00 0x7fa05013d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa05015d9a0 con 0x7fa050072740 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa05015d820 con 0x7fa0501454b0 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa05016ac80 con 0x7fa050145a00 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 0x7fa05015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 0x7fa05015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 0x7fa05015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:52216/0 (socket says 172.21.15.22:52216) 2024-04-27T07:52:40.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04f7fe640 1 -- 172.21.15.22:0/1772646408 learned_addr learned my addr 172.21.15.22:0/1772646408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:40.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa054ba4640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa050145a00 0x7fa05013d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 -- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa050145a00 msgr2=0x7fa05013d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa050145a00 0x7fa05013d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 -- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 msgr2=0x7fa05015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 0x7fa05015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 -- 172.21.15.22:0/1772646408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05013be10 con 0x7fa050072740 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04ffff640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 0x7fa05015bdc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c400 tx=0x7fa04400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044017070 con 0x7fa050072740 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa044005cf0 con 0x7fa050072740 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440072f0 con 0x7fa050072740 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05015bae0 con 0x7fa050072740 2024-04-27T07:52:40.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0500721d0 con 0x7fa050072740 2024-04-27T07:52:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050068c30 con 0x7fa050072740 2024-04-27T07:52:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.641+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa04401a070 con 0x7fa050072740 2024-04-27T07:52:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.645+0000 7fa04cff9640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 0x7fa03c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.645+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa044085c90 con 0x7fa050072740 2024-04-27T07:52:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.645+0000 7fa04f7fe640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 0x7fa03c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:40.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.645+0000 7fa04f7fe640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 0x7fa03c06a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa050073290 tx=0x7fa040003700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:40.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.649+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa044055bf0 con 0x7fa050072740 2024-04-27T07:52:40.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa05006f5d0 con 0x7fa050072740 2024-04-27T07:52:40.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa04cff9640 1 -- 172.21.15.22:0/1772646408 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7fa044059ef0 con 0x7fa050072740 2024-04-27T07:52:40.880 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:52:40.880 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","modified":"2024-04-27T07:49:58.411370Z","created":"2024-04-27T07:49:58.411370Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3301","nonce":0},{"type":"v1","addr":"172.21.15.183:6790","nonce":0}]},"addr":"172.21.15.183:6790/0","public_addr":"172.21.15.183:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-27T07:52:40.880 INFO:teuthology.orchestra.run.smithi022.stderr:dumped monmap epoch 1 2024-04-27T07:52:40.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 msgr2=0x7fa03c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 0x7fa03c06a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa050073290 tx=0x7fa040003700 comp rx=0 tx=0).stop 2024-04-27T07:52:40.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 msgr2=0x7fa05015bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:40.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 0x7fa05015bdc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c400 tx=0x7fa04400c8c0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 shutdown_connections 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa03c068380 0x7fa03c06a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa050145a00 0x7fa05013d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0501454b0 0x7fa05015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 --2- 172.21.15.22:0/1772646408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa050072740 0x7fa05015bdc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 >> 172.21.15.22:0/1772646408 conn(0x7fa050067f10 msgr2=0x7fa05013fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 shutdown_connections 2024-04-27T07:52:40.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:40.877+0000 7fa056607640 1 -- 172.21.15.22:0/1772646408 wait complete. 2024-04-27T07:52:40.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-27T07:52:41.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.025+0000 7f4380d89640 1 Processor -- start 2024-04-27T07:52:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.025+0000 7f4380d89640 1 -- start start 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4380d89640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f437c160cc0 0x7f437c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4380d89640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f437c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4380d89640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c161210 con 0x7f437c15c720 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4380d89640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f437c06c5a0 con 0x7f437c160cc0 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4380d89640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f437c06bc70 con 0x7f437c0add70 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437a59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f437c15c720 0x7f437c15cb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54926/0 (socket says 172.21.15.22:54926) 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437a59c640 1 -- 172.21.15.22:0/2482198153 learned_addr learned my addr 172.21.15.22:0/2482198153 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:52:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437ad9d640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f437c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 --2- 172.21.15.22:0/2482198153 >> v2:172.21.15.183:3300/0 conn(0x7f437c160cc0 0x7f437c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551270673 0 0) 0x7f437c161210 con 0x7f437c15c720 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003530 con 0x7f437c15c720 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 584379396 0 0) 0x7f4364003530 con 0x7f437c15c720 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 msgr2=0x7f437c0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f437c0ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 >> v2:172.21.15.183:3300/0 conn(0x7f437c160cc0 msgr2=0x7f437c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 --2- 172.21.15.22:0/2482198153 >> v2:172.21.15.183:3300/0 conn(0x7f437c160cc0 0x7f437c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c06bf50 con 0x7f437c15c720 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f4370001680 con 0x7f437c15c720 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 >> v1:172.21.15.22:6789/0 conn(0x7f437c15c720 legacy=0x7f437c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43640039c0 0x7f4364003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4364004810 0x7f4364004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f43640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f437c06c5a0 con 0x7f4364004810 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f437c06bc70 con 0x7f43640039c0 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f436400b9d0 con 0x7f437c0add70 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4364004810 0x7f4364004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437a59c640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43640039c0 0x7f4364003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f437ad9d640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f43640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 -- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 msgr2=0x7f43640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c0add70 0x7f43640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 -- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43640039c0 msgr2=0x7f4364003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:52:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43640039c0 0x7f4364003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.029+0000 7f4379d9b640 1 -- 172.21.15.22:0/2482198153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364003530 con 0x7f4364004810 2024-04-27T07:52:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4364004810 0x7f4364004c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ae0 tx=0x7f436c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b0c71e18fe43a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b2070 con 0x7f4364004810 2024-04-27T07:52:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c0a55c0 con 0x7f4364004810 2024-04-27T07:52:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437959a640 1 -- 172.21.15.22:0/2482198153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0096b0 con 0x7f4364004810 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/2482198153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4364004810 msgr2=0x7f4364004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 --2- 172.21.15.22:0/2482198153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4364004810 0x7f4364004c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ae0 tx=0x7f436c01ca50 comp rx=0 tx=0).stop 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437ad9d640 1 -- 172.21.15.22:0/2482198153 reap_dead start 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/2482198153 shutdown_connections 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/2482198153 >> 172.21.15.22:0/2482198153 conn(0x7f437c0b3470 msgr2=0x7f437c0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/2482198153 shutdown_connections 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/2482198153 wait complete. 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 Processor -- start 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- start start 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4364004810 0x7f437c1437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c070420 0x7f437c15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0b6030 con 0x7f437c06ffa0 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0b5eb0 con 0x7f4364004810 2024-04-27T07:52:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f437c15d210 con 0x7f437c070420 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48030/0 (socket says 172.21.15.22:48030) 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 -- 172.21.15.22:0/1322526632 learned_addr learned my addr 172.21.15.22:0/1322526632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437a59c640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4364004810 0x7f437c1437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f437ad9d640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c070420 0x7f437c15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 -- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c070420 msgr2=0x7f437c15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c070420 0x7f437c15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 -- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4364004810 msgr2=0x7f437c1437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4364004810 0x7f437c1437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 -- 172.21.15.22:0/1322526632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c1f55e0 con 0x7f437c06ffa0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4379d9b640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f436c01c880 tx=0x7f4364005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0bc030 con 0x7f437c06ffa0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c00f000 con 0x7f437c06ffa0 2024-04-27T07:52:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c06f4c0 con 0x7f437c06ffa0 2024-04-27T07:52:41.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b2040 con 0x7f437c06ffa0 2024-04-27T07:52:41.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.033+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c143d00 con 0x7f437c06ffa0 2024-04-27T07:52:41.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c069340 con 0x7f437c06ffa0 2024-04-27T07:52:41.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f436c0bf070 con 0x7f437c06ffa0 2024-04-27T07:52:41.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f436affd640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 0x7f435c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:52:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f436c120420 con 0x7f437c06ffa0 2024-04-27T07:52:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f437a59c640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 0x7f435c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:52:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.037+0000 7f437a59c640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 0x7f435c06a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4370001060 tx=0x7f4370009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:52:41.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.041+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f436c0167c0 con 0x7f437c06ffa0 2024-04-27T07:52:41.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.257+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f437c0b4300 con 0x7f437c06ffa0 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.257+0000 7f436affd640 1 -- 172.21.15.22:0/1322526632 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v11) ==== 82+0+151 (secure 0 0 0) 0x7f436c0f0380 con 0x7f437c06ffa0 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout:[client.admin] 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAmrixmaWM0JBAAb/2DeNmyAcgAupnC1hDySA== 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow *" 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: caps mgr = "allow *" 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow *" 2024-04-27T07:52:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow *" 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 msgr2=0x7f435c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 0x7f435c06a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4370001060 tx=0x7f4370009040 comp rx=0 tx=0).stop 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 msgr2=0x7f437c15f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f436c01c880 tx=0x7f4364005840 comp rx=0 tx=0).stop 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 shutdown_connections 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f435c068380 0x7f435c06a840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f437c070420 0x7f437c15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f437c06ffa0 0x7f437c15f2e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 --2- 172.21.15.22:0/1322526632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4364004810 0x7f437c1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:52:41.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 >> 172.21.15.22:0/1322526632 conn(0x7f437c0b3470 msgr2=0x7f437c0b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:52:41.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 shutdown_connections 2024-04-27T07:52:41.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:52:41.261+0000 7f4380d89640 1 -- 172.21.15.22:0/1322526632 wait complete. 2024-04-27T07:52:41.280 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-04-27T07:52:41.286 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.vSsz0jpzuL 2024-04-27T07:52:41.286 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:52:41.286 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/tmp/tmp.vSsz0jpzuL 2024-04-27T07:52:41.332 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:52:41.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/tmp/tmp.BsSpkJ5lk2xfstests-dev/local.config 2024-04-27T07:52:41.387 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.ywDIeOSfVMtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.ZBGWbtclM6scratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.22:6789 -k /tmp/tmp.vSsz0jpzuL --client_mountpoint /test" 2024-04-27T07:52:41.387 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:52:41.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/tmp/tmp.BsSpkJ5lk2xfstests-dev/ceph.exclude 2024-04-27T07:52:41.443 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.BsSpkJ5lk2xfstests-dev && exec sudo make) 2024-04-27T07:52:41.504 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize -cfi 2024-04-27T07:52:41.807 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-27T07:52:41.807 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './config.guess' 2024-04-27T07:52:41.812 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './config.sub' 2024-04-27T07:52:41.821 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './install-sh' 2024-04-27T07:52:41.828 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './ltmain.sh' 2024-04-27T07:52:41.850 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-27T07:52:41.850 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-27T07:52:41.862 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-27T07:52:41.868 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-27T07:52:41.873 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-27T07:52:41.878 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-27T07:52:41.884 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-27T07:52:41.885 INFO:teuthology.orchestra.run.smithi022.stdout:cp include/install-sh . 2024-04-27T07:52:41.886 INFO:teuthology.orchestra.run.smithi022.stdout:aclocal -I m4 2024-04-27T07:52:43.072 INFO:teuthology.orchestra.run.smithi022.stdout:autoheader 2024-04-27T07:52:43.438 INFO:teuthology.orchestra.run.smithi022.stdout:autoconf 2024-04-27T07:52:43.861 INFO:teuthology.orchestra.run.smithi022.stdout:./configure \ 2024-04-27T07:52:43.862 INFO:teuthology.orchestra.run.smithi022.stdout: --libexecdir=/usr/lib \ 2024-04-27T07:52:43.862 INFO:teuthology.orchestra.run.smithi022.stdout: --exec_prefix=/var/lib 2024-04-27T07:52:44.280 INFO:teuthology.orchestra.run.smithi022.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-27T07:52:44.281 INFO:teuthology.orchestra.run.smithi022.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-27T07:52:44.317 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to print strings... printf 2024-04-27T07:52:44.317 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc... gcc 2024-04-27T07:52:44.494 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the C compiler works... yes 2024-04-27T07:52:44.494 INFO:teuthology.orchestra.run.smithi022.stdout:checking for C compiler default output file name... a.out 2024-04-27T07:52:44.535 INFO:teuthology.orchestra.run.smithi022.stdout:checking for suffix of executables... 2024-04-27T07:52:44.627 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether we are cross compiling... no 2024-04-27T07:52:44.647 INFO:teuthology.orchestra.run.smithi022.stdout:checking for suffix of object files... o 2024-04-27T07:52:44.665 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the compiler supports GNU C... yes 2024-04-27T07:52:44.683 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc accepts -g... yes 2024-04-27T07:52:44.801 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to enable C11 features... none needed 2024-04-27T07:52:44.806 INFO:teuthology.orchestra.run.smithi022.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-27T07:52:44.810 INFO:teuthology.orchestra.run.smithi022.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-27T07:52:44.812 INFO:teuthology.orchestra.run.smithi022.stdout:checking for egrep... /usr/bin/grep -E 2024-04-27T07:52:44.813 INFO:teuthology.orchestra.run.smithi022.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-27T07:52:44.820 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-27T07:52:44.822 INFO:teuthology.orchestra.run.smithi022.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-27T07:52:44.835 INFO:teuthology.orchestra.run.smithi022.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-27T07:52:44.859 INFO:teuthology.orchestra.run.smithi022.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-27T07:52:44.859 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether ln -s works... yes 2024-04-27T07:52:44.871 INFO:teuthology.orchestra.run.smithi022.stdout:checking the maximum length of command line arguments... 1572864 2024-04-27T07:52:44.876 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-27T07:52:44.876 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-27T07:52:44.876 INFO:teuthology.orchestra.run.smithi022.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-27T07:52:44.876 INFO:teuthology.orchestra.run.smithi022.stdout:checking for objdump... objdump 2024-04-27T07:52:44.877 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to recognize dependent libraries... pass_all 2024-04-27T07:52:44.877 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dlltool... no 2024-04-27T07:52:44.877 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-27T07:52:44.877 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ar... ar 2024-04-27T07:52:44.907 INFO:teuthology.orchestra.run.smithi022.stdout:checking for archiver @FILE support... @ 2024-04-27T07:52:44.907 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strip... strip 2024-04-27T07:52:44.907 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ranlib... ranlib 2024-04-27T07:52:44.907 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gawk... gawk 2024-04-27T07:52:44.979 INFO:teuthology.orchestra.run.smithi022.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-27T07:52:44.981 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sysroot... no 2024-04-27T07:52:44.985 INFO:teuthology.orchestra.run.smithi022.stdout:checking for a working dd... /usr/bin/dd 2024-04-27T07:52:44.993 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-27T07:52:45.004 INFO:teuthology.orchestra.run.smithi022.stdout:checking for mt... mt 2024-04-27T07:52:45.012 INFO:teuthology.orchestra.run.smithi022.stdout:checking if mt is a manifest tool... no 2024-04-27T07:52:45.035 INFO:teuthology.orchestra.run.smithi022.stdout:checking for stdio.h... yes 2024-04-27T07:52:45.087 INFO:teuthology.orchestra.run.smithi022.stdout:checking for stdlib.h... yes 2024-04-27T07:52:45.123 INFO:teuthology.orchestra.run.smithi022.stdout:checking for string.h... yes 2024-04-27T07:52:45.158 INFO:teuthology.orchestra.run.smithi022.stdout:checking for inttypes.h... yes 2024-04-27T07:52:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:checking for stdint.h... yes 2024-04-27T07:52:45.218 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strings.h... yes 2024-04-27T07:52:45.252 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/stat.h... yes 2024-04-27T07:52:45.280 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/types.h... yes 2024-04-27T07:52:45.328 INFO:teuthology.orchestra.run.smithi022.stdout:checking for unistd.h... yes 2024-04-27T07:52:45.366 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dlfcn.h... yes 2024-04-27T07:52:45.369 INFO:teuthology.orchestra.run.smithi022.stdout:checking for objdir... .libs 2024-04-27T07:52:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-27T07:52:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-27T07:52:45.477 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-27T07:52:45.743 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc static flag -static works... yes 2024-04-27T07:52:45.776 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -c -o file.o... yes 2024-04-27T07:52:45.776 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-27T07:52:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-27T07:52:45.816 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether -lc should be explicitly linked in... no 2024-04-27T07:52:45.899 INFO:teuthology.orchestra.run.smithi022.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-27T07:52:45.899 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to hardcode library paths into programs... immediate 2024-04-27T07:52:45.912 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether stripping libraries is possible... yes 2024-04-27T07:52:45.912 INFO:teuthology.orchestra.run.smithi022.stdout:checking if libtool supports shared libraries... yes 2024-04-27T07:52:45.912 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether to build shared libraries... yes 2024-04-27T07:52:45.912 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether to build static libraries... yes 2024-04-27T07:52:45.922 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc... (cached) gcc 2024-04-27T07:52:45.950 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-04-27T07:52:45.950 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-27T07:52:45.950 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-04-27T07:52:45.950 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gmake... /usr/bin/gmake 2024-04-27T07:52:45.950 INFO:teuthology.orchestra.run.smithi022.stderr:./configure: line 12753: test: $(SHELL): binary operator expected 2024-04-27T07:52:45.951 INFO:teuthology.orchestra.run.smithi022.stdout:checking for tar... /bin/tar 2024-04-27T07:52:45.951 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gzip... /bin/gzip 2024-04-27T07:52:45.962 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc -MM is supported... yes 2024-04-27T07:52:45.963 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sort... /bin/sort 2024-04-27T07:52:45.963 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether ln -s works... yes 2024-04-27T07:52:45.964 INFO:teuthology.orchestra.run.smithi022.stdout:checking for rpm... no 2024-04-27T07:52:45.999 INFO:teuthology.orchestra.run.smithi022.stdout:checking for assert.h... yes 2024-04-27T07:52:46.033 INFO:teuthology.orchestra.run.smithi022.stdout:checking for bstring.h... no 2024-04-27T07:52:46.065 INFO:teuthology.orchestra.run.smithi022.stdout:checking for libgen.h... yes 2024-04-27T07:52:46.147 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dirent.h... yes 2024-04-27T07:52:46.234 INFO:teuthology.orchestra.run.smithi022.stdout:checking for errno.h... yes 2024-04-27T07:52:46.272 INFO:teuthology.orchestra.run.smithi022.stdout:checking for malloc.h... yes 2024-04-27T07:52:46.312 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid.h... no 2024-04-27T07:52:46.349 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid/uuid.h... yes 2024-04-27T07:52:46.386 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/uuid.h... no 2024-04-27T07:52:46.421 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/file.h... yes 2024-04-27T07:52:46.455 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fcntl.h... yes 2024-04-27T07:52:46.496 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/syssgi.h... no 2024-04-27T07:52:46.552 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/param.h... yes 2024-04-27T07:52:46.552 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/stat.h... (cached) yes 2024-04-27T07:52:46.587 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/statvfs.h... yes 2024-04-27T07:52:46.626 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/time.h... yes 2024-04-27T07:52:46.663 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/ioctl.h... yes 2024-04-27T07:52:46.698 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/wait.h... yes 2024-04-27T07:52:46.698 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/types.h... (cached) yes 2024-04-27T07:52:46.698 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strings.h... (cached) yes 2024-04-27T07:52:46.733 INFO:teuthology.orchestra.run.smithi022.stdout:checking for err.h... yes 2024-04-27T07:52:46.767 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/falloc.h... yes 2024-04-27T07:52:46.803 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-27T07:52:46.845 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-27T07:52:46.888 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/platform_defs.h... no 2024-04-27T07:52:46.927 INFO:teuthology.orchestra.run.smithi022.stdout:checking for btrfs/ioctl.h... no 2024-04-27T07:52:46.970 INFO:teuthology.orchestra.run.smithi022.stdout:checking for cifs/ioctl.h... no 2024-04-27T07:52:47.005 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/mman.h... yes 2024-04-27T07:52:47.038 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/ext4.h... no 2024-04-27T07:52:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xfs_log_format.h... no 2024-04-27T07:52:47.103 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid_compare... no 2024-04-27T07:52:47.141 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid_compare in -luuid... yes 2024-04-27T07:52:47.239 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xfs.h... yes 2024-04-27T07:52:47.257 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/libxfs.h... no 2024-04-27T07:52:47.298 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc options needed to detect all undeclared functions... none needed 2024-04-27T07:52:47.318 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-27T07:52:47.368 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xqm.h... yes 2024-04-27T07:52:47.439 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-27T07:52:47.473 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/handle.h... yes 2024-04-27T07:52:47.514 INFO:teuthology.orchestra.run.smithi022.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-27T07:52:47.554 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/acl.h... yes 2024-04-27T07:52:47.595 INFO:teuthology.orchestra.run.smithi022.stdout:checking for acl/libacl.h... yes 2024-04-27T07:52:47.637 INFO:teuthology.orchestra.run.smithi022.stdout:checking for acl_init in -lacl... yes 2024-04-27T07:52:47.669 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gdbm-ndbm.h... no 2024-04-27T07:52:47.709 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gdbm/ndbm.h... no 2024-04-27T07:52:47.746 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ndbm.h... no 2024-04-27T07:52:47.783 INFO:teuthology.orchestra.run.smithi022.stdout:checking for libaio.h... yes 2024-04-27T07:52:47.858 INFO:teuthology.orchestra.run.smithi022.stdout:checking for liburing.h... yes 2024-04-27T07:52:47.902 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/capability.h... yes 2024-04-27T07:52:47.946 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/fiemap.h... yes 2024-04-27T07:52:47.991 INFO:teuthology.orchestra.run.smithi022.stdout:checking for fallocate... yes 2024-04-27T07:52:48.040 INFO:teuthology.orchestra.run.smithi022.stdout:checking for open_by_handle_at... yes 2024-04-27T07:52:48.078 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/fs.h... yes 2024-04-27T07:52:48.119 INFO:teuthology.orchestra.run.smithi022.stdout:checking for btrfsutil.h... no 2024-04-27T07:52:48.165 INFO:teuthology.orchestra.run.smithi022.stdout:checking for copy_file_range... yes 2024-04-27T07:52:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:checking for SEEK_DATA... yes 2024-04-27T07:52:48.284 INFO:teuthology.orchestra.run.smithi022.stdout:checking for BMV_OF_SHARED... yes 2024-04-27T07:52:48.327 INFO:teuthology.orchestra.run.smithi022.stdout:checking for nftw... yes 2024-04-27T07:52:48.365 INFO:teuthology.orchestra.run.smithi022.stdout:checking for RLIMIT_NOFILE... yes 2024-04-27T07:52:48.407 INFO:teuthology.orchestra.run.smithi022.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi022.stdout:checking for FICLONE... yes 2024-04-27T07:52:48.489 INFO:teuthology.orchestra.run.smithi022.stdout:checking for renameat2... yes 2024-04-27T07:52:48.524 INFO:teuthology.orchestra.run.smithi022.stdout:checking for reallocarray... yes 2024-04-27T07:52:48.557 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct mount_attr... yes 2024-04-27T07:52:48.605 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-27T07:52:48.642 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-27T07:52:48.677 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-27T07:52:48.712 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-27T07:52:48.745 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-27T07:52:48.780 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-27T07:52:48.817 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-27T07:52:48.849 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/btrfs.h... yes 2024-04-27T07:52:48.882 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/btrfs_tree.h... yes 2024-04-27T07:52:48.901 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-27T07:52:48.926 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-27T07:52:48.945 INFO:teuthology.orchestra.run.smithi022.stdout:configure: creating ./config.status 2024-04-27T07:52:49.368 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: creating include/builddefs 2024-04-27T07:52:49.384 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: creating include/config.h 2024-04-27T07:52:49.396 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: executing libtool commands 2024-04-27T07:52:49.447 INFO:teuthology.orchestra.run.smithi022.stdout:Building include 2024-04-27T07:52:49.451 INFO:teuthology.orchestra.run.smithi022.stdout:Building lib 2024-04-27T07:52:49.455 INFO:teuthology.orchestra.run.smithi022.stdout: [LTDEP] 2024-04-27T07:52:49.615 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dataascii.lo 2024-04-27T07:52:49.736 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] databin.lo 2024-04-27T07:52:49.902 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] datapid.lo 2024-04-27T07:52:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] file_lock.lo 2024-04-27T07:52:50.202 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] forker.lo 2024-04-27T07:52:50.363 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] pattern.lo 2024-04-27T07:52:50.488 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] open_flags.lo 2024-04-27T07:52:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] random_range.lo 2024-04-27T07:52:50.907 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] string_to_tokens.lo 2024-04-27T07:52:51.007 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] str_to_bytes.lo 2024-04-27T07:52:51.160 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] tlibio.lo 2024-04-27T07:52:51.435 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] write_log.lo 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c: In function 'wlog_record_write': 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c:205:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c:209:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2024-04-27T07:52:51.503 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:51.710 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] random.lo 2024-04-27T07:52:51.819 INFO:teuthology.orchestra.run.smithi022.stdout: [LD] 2024-04-27T07:52:51.897 INFO:teuthology.orchestra.run.smithi022.stdout:Building ltp 2024-04-27T07:52:51.900 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:52.058 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] doio 2024-04-27T07:52:52.802 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsstress 2024-04-27T07:52:54.343 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsx 2024-04-27T07:52:55.153 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] growfiles 2024-04-27T07:52:55.727 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] iogen 2024-04-27T07:52:55.844 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c: In function 'main': 2024-04-27T07:52:55.844 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c:280:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:52:55.844 INFO:teuthology.orchestra.run.smithi022.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2024-04-27T07:52:55.844 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c: In function 'parse_cmdline': 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c:1535:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr: 1535 | getcwd(fptr->f_path, 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr: 1536 | sizeof(fptr->f_path)-1); 2024-04-27T07:52:55.847 INFO:teuthology.orchestra.run.smithi022.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:56.145 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-stress 2024-04-27T07:52:56.283 INFO:teuthology.orchestra.run.smithi022.stderr:aio-stress.c: In function 'setup_shared_mem': 2024-04-27T07:52:56.283 INFO:teuthology.orchestra.run.smithi022.stderr:aio-stress.c:1013:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:52:56.283 INFO:teuthology.orchestra.run.smithi022.stderr: 1013 | ftruncate(fd, total_ram); 2024-04-27T07:52:56.283 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:52:56.627 INFO:teuthology.orchestra.run.smithi022.stdout:Building src 2024-04-27T07:52:56.635 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:57.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:57.372 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:58.868 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:58.877 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:59.157 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2024-04-27T07:52:59.184 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirstress 2024-04-27T07:52:59.442 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fill 2024-04-27T07:52:59.611 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fill2 2024-04-27T07:52:59.858 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] getpagesize 2024-04-27T07:52:59.987 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] holes 2024-04-27T07:53:00.179 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] lstat64 2024-04-27T07:53:00.355 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] nametest 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c: In function 'main': 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:236:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: 236 | write(1, "\n", 1); 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:238:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: 238 | write(1, ".", 1); 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:288:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: 288 | write(1, ".", 1); 2024-04-27T07:53:00.475 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:00.604 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] permname 2024-04-27T07:53:00.811 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] randholes 2024-04-27T07:53:01.070 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] runas 2024-04-27T07:53:01.249 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] truncfile 2024-04-27T07:53:01.438 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] usemem 2024-04-27T07:53:01.578 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmapcat 2024-04-27T07:53:01.677 INFO:teuthology.orchestra.run.smithi022.stderr:mmapcat.c: In function 'main': 2024-04-27T07:53:01.678 INFO:teuthology.orchestra.run.smithi022.stderr:mmapcat.c:29:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:01.678 INFO:teuthology.orchestra.run.smithi022.stderr: 29 | write(1,ptr2,ptr - ptr2); 2024-04-27T07:53:01.678 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:01.711 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] append_reader 2024-04-27T07:53:01.859 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] append_writer 2024-04-27T07:53:01.998 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirperf 2024-04-27T07:53:02.227 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] metaperf 2024-04-27T07:53:02.344 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'crfiles': 2024-04-27T07:53:02.344 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:166:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.344 INFO:teuthology.orchestra.run.smithi022.stderr: 166 | write(fd, buf, fsize); 2024-04-27T07:53:02.344 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 't_linkun': 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:454:25: warning: ignoring return value of 'link' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr: 454 | link("a", *fnp); 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~ 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 't_chown': 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:404:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.345 INFO:teuthology.orchestra.run.smithi022.stderr: 404 | chown(*fnp, 2, -1); 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:406:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: 406 | chown(*fnp, 1, -1); 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'i_chown': 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:291:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: 291 | chown(*fnp, 1, -1); 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'main': 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:152:9: warning: ignoring return value of 'chdir' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: 152 | chdir(".."); 2024-04-27T07:53:02.346 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~ 2024-04-27T07:53:02.524 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] devzero 2024-04-27T07:53:02.697 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] feature 2024-04-27T07:53:02.943 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] alloc 2024-04-27T07:53:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:alloc.c: In function 'main': 2024-04-27T07:53:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:alloc.c:247:17: warning: ignoring return value of 'mkstemp' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:03.073 INFO:teuthology.orchestra.run.smithi022.stderr: 247 | mkstemp(tmpfile); 2024-04-27T07:53:03.073 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:03.172 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fault 2024-04-27T07:53:03.354 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fstest 2024-04-27T07:53:03.625 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_access_root 2024-04-27T07:53:03.731 INFO:teuthology.orchestra.run.smithi022.stderr:t_access_root.c: In function 'main': 2024-04-27T07:53:03.731 INFO:teuthology.orchestra.run.smithi022.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:03.731 INFO:teuthology.orchestra.run.smithi022.stderr: 55 | system(cmd); 2024-04-27T07:53:03.731 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~ 2024-04-27T07:53:03.779 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] godown 2024-04-27T07:53:03.956 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] resvtest 2024-04-27T07:53:04.149 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] writemod 2024-04-27T07:53:04.286 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] writev_on_pagefault 2024-04-27T07:53:04.438 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] makeextents 2024-04-27T07:53:04.613 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] itrash 2024-04-27T07:53:04.767 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] rename 2024-04-27T07:53:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] multi_open_unlink 2024-04-27T07:53:05.054 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] unwritten_sync 2024-04-27T07:53:05.243 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] genhashnames 2024-04-27T07:53:05.427 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_holes 2024-04-27T07:53:05.576 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_writev 2024-04-27T07:53:05.718 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_truncate_cmtime 2024-04-27T07:53:05.878 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirhash_collide 2024-04-27T07:53:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_rename_overwrite 2024-04-27T07:53:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] holetest 2024-04-27T07:53:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:holetest.c: In function 'pt_write_marker': 2024-04-27T07:53:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:holetest.c:140:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2024-04-27T07:53:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_truncate_self 2024-04-27T07:53:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] af_unix 2024-04-27T07:53:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_stale_pmd 2024-04-27T07:53:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c: In function 'main': 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:42:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: 42 | ftruncate(fd, 0); 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:43:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: 43 | ftruncate(fd, MiB(4)); 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:55:9: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2024-04-27T07:53:06.829 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:06.867 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_cow_race 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c: In function 'main': 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr: 61 | ftruncate(fd, 0); 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c:62:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:06.990 INFO:teuthology.orchestra.run.smithi022.stderr: 62 | ftruncate(fd, MiB(2)); 2024-04-27T07:53:06.991 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_fallocate 2024-04-27T07:53:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsync-err 2024-04-27T07:53:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_write_ro 2024-04-27T07:53:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-27T07:53:07.644 INFO:teuthology.orchestra.run.smithi022.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2024-04-27T07:53:07.644 INFO:teuthology.orchestra.run.smithi022.stderr:t_ext4_dax_journal_corruption.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:07.644 INFO:teuthology.orchestra.run.smithi022.stderr: 61 | ftruncate(fd, 0); 2024-04-27T07:53:07.644 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-27T07:53:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ofd_locks 2024-04-27T07:53:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_collision 2024-04-27T07:53:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmap-write-concurrent 2024-04-27T07:53:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_get_file_time 2024-04-27T07:53:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_create_short_dirs 2024-04-27T07:53:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_create_long_dirs 2024-04-27T07:53:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_enospc 2024-04-27T07:53:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_writev_overlap 2024-04-27T07:53:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] checkpoint_journal 2024-04-27T07:53:09.480 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmap-rw-fault 2024-04-27T07:53:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] allocstale 2024-04-27T07:53:09.845 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_cow_memory_failure 2024-04-27T07:53:10.014 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fake-dump-rootino 2024-04-27T07:53:10.156 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-buf-fault 2024-04-27T07:53:10.291 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] rewinddir-test 2024-04-27T07:53:10.477 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] readdir-while-renames 2024-04-27T07:53:10.667 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] xfsctl 2024-04-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bstat 2024-04-27T07:53:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mtab 2024-04-27T07:53:11.215 INFO:teuthology.orchestra.run.smithi022.stderr:t_mtab.c: In function 'update_mtab': 2024-04-27T07:53:11.215 INFO:teuthology.orchestra.run.smithi022.stderr:t_mtab.c:256:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:11.215 INFO:teuthology.orchestra.run.smithi022.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2024-04-27T07:53:11.215 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:11.296 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] getdevicesize 2024-04-27T07:53:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] preallo_rw_pattern_reader 2024-04-27T07:53:11.586 INFO:teuthology.orchestra.run.smithi022.stderr:In file included from preallo_rw_pattern_reader.c:2: 2024-04-27T07:53:11.586 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c: In function 'main': 2024-04-27T07:53:11.586 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c:38:4: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:11.586 INFO:teuthology.orchestra.run.smithi022.stderr: 38 | read(fd, &x, 1048576); 2024-04-27T07:53:11.587 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:11.625 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] preallo_rw_pattern_writer 2024-04-27T07:53:11.745 INFO:teuthology.orchestra.run.smithi022.stderr:In file included from preallo_rw_pattern_writer.c:2: 2024-04-27T07:53:11.745 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c: In function 'main': 2024-04-27T07:53:11.745 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c:34:4: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:11.745 INFO:teuthology.orchestra.run.smithi022.stderr: 34 | write(fd, &x, 1048576); 2024-04-27T07:53:11.745 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:11.778 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] ftrunc 2024-04-27T07:53:11.928 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] trunc 2024-04-27T07:53:12.089 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fs_perms 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'main': 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr: 54 | system("rm -f test.file"); 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'testsetup': 2024-04-27T07:53:12.198 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: 67 | system("cp testx.file test.file"); 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: 69 | chown("test.file",cuserId,cgroupId); 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'testfperm': 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: 123 | seteuid(0); 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~ 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: 124 | setegid(0); 2024-04-27T07:53:12.199 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~ 2024-04-27T07:53:12.247 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] testx 2024-04-27T07:53:12.363 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] looptest 2024-04-27T07:53:12.553 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] locktest 2024-04-27T07:53:12.896 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] unwritten_mmap 2024-04-27T07:53:13.063 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_unlink_test 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c: In function 'main': 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c:106:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c:178:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr: 178 | system(fname); 2024-04-27T07:53:13.188 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~ 2024-04-27T07:53:13.285 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] deduperace 2024-04-27T07:53:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_unlink_test_modified 2024-04-27T07:53:13.703 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:83:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr: 83 | write(fd, fname, sizeof(fname)); 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:139:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr: 139 | write(fd, fname, sizeof(fname)); 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:185:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:13.704 INFO:teuthology.orchestra.run.smithi022.stderr: 185 | system(fname); 2024-04-27T07:53:13.705 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~ 2024-04-27T07:53:13.789 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_offset 2024-04-27T07:53:13.937 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_futimens 2024-04-27T07:53:14.079 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_immutable 2024-04-27T07:53:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'create_test_area': 2024-04-27T07:53:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.220 INFO:teuthology.orchestra.run.smithi022.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2024-04-27T07:53:14.221 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.221 INFO:teuthology.orchestra.run.smithi022.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2024-04-27T07:53:14.221 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'remove_test_area': 2024-04-27T07:53:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.224 INFO:teuthology.orchestra.run.smithi022.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'check_test_area': 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: 1788 | asprintf(&path, "%s/", dir); 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2024-04-27T07:53:14.226 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.227 INFO:teuthology.orchestra.run.smithi022.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-27T07:53:14.228 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'test_immutable': 2024-04-27T07:53:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.233 INFO:teuthology.orchestra.run.smithi022.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:408:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: 408 | chown(path, 0, 0); 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2024-04-27T07:53:14.234 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:553:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: 553 | chown(path, 0, 0); 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.235 INFO:teuthology.orchestra.run.smithi022.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.236 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: 653 | chown(path, 0, 0); 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:705:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: 705 | chown(path, 0, 0); 2024-04-27T07:53:14.237 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'test_append': 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.238 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1051:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: 1051 | chown(path, 0, 0); 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1516:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: 1516 | chown(path, 0, 0); 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.239 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.240 INFO:teuthology.orchestra.run.smithi022.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1573:11: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.241 INFO:teuthology.orchestra.run.smithi022.stderr: 1618 | chown(path, 0, 0); 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: 1644 | chown(path, 0, 0); 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.242 INFO:teuthology.orchestra.run.smithi022.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1692:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr: 1692 | chown(path, 0, 0); 2024-04-27T07:53:14.243 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.244 INFO:teuthology.orchestra.run.smithi022.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-27T07:53:14.244 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] stale_handle 2024-04-27T07:53:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:stale_handle.c: In function 'main': 2024-04-27T07:53:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:stale_handle.c:102:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:14.917 INFO:teuthology.orchestra.run.smithi022.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2024-04-27T07:53:14.917 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:14.971 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] pwrite_mmap_blocked 2024-04-27T07:53:15.137 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_offset2 2024-04-27T07:53:15.348 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] seek_sanity_test 2024-04-27T07:53:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'test18': 2024-04-27T07:53:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:460:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.484 INFO:teuthology.orchestra.run.smithi022.stderr: 460 | ftruncate(fd, 0); 2024-04-27T07:53:15.484 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.486 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'test_basic_support': 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:1207:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr: 1207 | ftruncate(fd, 0); 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:1256:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr: 1256 | ftruncate(fd, 0); 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.487 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:88:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: 88 | ftruncate(fd, 0); 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:89:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: 89 | pwrite(fd, "a", 1, offset); 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:98:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: 98 | ftruncate(fd, 0); 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:99:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: 99 | pwrite(fd, "a", 1, offset); 2024-04-27T07:53:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:15.824 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] seek_copy_test 2024-04-27T07:53:16.008 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_1 2024-04-27T07:53:16.116 INFO:teuthology.orchestra.run.smithi022.stderr:t_readdir_1.c: In function 'main': 2024-04-27T07:53:16.116 INFO:teuthology.orchestra.run.smithi022.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:16.116 INFO:teuthology.orchestra.run.smithi022.stderr: 30 | read(fd, buf, 100); 2024-04-27T07:53:16.116 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:16.154 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_2 2024-04-27T07:53:16.303 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsync-tester 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c: In function 'main': 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:391:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr: 391 | read(fd, &seed, sizeof(seed)); 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:541:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr: 541 | system("reboot -fn"); 2024-04-27T07:53:16.405 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c: In function 'drop_all_caches': 2024-04-27T07:53:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:90:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:16.406 INFO:teuthology.orchestra.run.smithi022.stderr: 90 | write(fd, value, sizeof(value)-1); 2024-04-27T07:53:16.406 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:16.563 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] nsexec 2024-04-27T07:53:16.757 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] cloner 2024-04-27T07:53:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] renameat2 2024-04-27T07:53:17.110 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_getcwd 2024-04-27T07:53:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:t_getcwd.c: In function 'test_getcwd': 2024-04-27T07:53:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:t_getcwd.c:21:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:17.227 INFO:teuthology.orchestra.run.smithi022.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2024-04-27T07:53:17.227 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:17.274 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] e4compact 2024-04-27T07:53:17.498 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] test-nextquota 2024-04-27T07:53:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] punch-alternating 2024-04-27T07:53:17.836 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-27T07:53:18.006 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] listxattr 2024-04-27T07:53:18.160 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-interleaved 2024-04-27T07:53:18.314 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_type 2024-04-27T07:53:18.467 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-invalidate-cache 2024-04-27T07:53:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:dio-invalidate-cache.c: In function 'main': 2024-04-27T07:53:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:dio-invalidate-cache.c:286:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:18.579 INFO:teuthology.orchestra.run.smithi022.stderr: 286 | write(fd, buf, blksz * nr_child); 2024-04-27T07:53:18.579 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:18.680 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] stat_test 2024-04-27T07:53:19.003 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_encrypted_d_revalidate 2024-04-27T07:53:19.175 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] attr_replace_test 2024-04-27T07:53:19.352 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] swapon 2024-04-27T07:53:19.510 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mkswap 2024-04-27T07:53:19.661 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_attr_corruption 2024-04-27T07:53:19.805 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_open_tmpfiles 2024-04-27T07:53:19.982 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fscrypt-crypt-util 2024-04-27T07:53:20.681 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_null_ocount 2024-04-27T07:53:20.846 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] splice-test 2024-04-27T07:53:21.037 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] chprojid_fail 2024-04-27T07:53:21.206 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] detached_mounts_propagation 2024-04-27T07:53:21.401 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] ext4_resize 2024-04-27T07:53:21.556 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_3 2024-04-27T07:53:21.757 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] splice2pipe 2024-04-27T07:53:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c: In function 'prepare_pipe': 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c:57:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr: 57 | write(p[1], buffer, n); 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c:65:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr: 65 | read(p[0], buffer, n); 2024-04-27T07:53:21.871 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:21.923 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] uuid_ioctl 2024-04-27T07:53:22.088 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-27T07:53:22.256 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fiemap-fault 2024-04-27T07:53:22.403 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fiemap-tester 2024-04-27T07:53:22.673 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_stripealign 2024-04-27T07:53:22.822 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] open_by_handle 2024-04-27T07:53:23.076 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_dio 2024-04-27T07:53:23.248 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] uring_read_fault 2024-04-27T07:53:23.446 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] xfsfind 2024-04-27T07:53:23.657 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_reflink_read_race 2024-04-27T07:53:23.864 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fssum 2024-04-27T07:53:24.355 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2024-04-27T07:53:24.363 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] replay-log 2024-04-27T07:53:24.726 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2024-04-27T07:53:24.729 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2024-04-27T07:53:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aiocp 2024-04-27T07:53:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-27T07:53:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-append-write-read-race 2024-04-27T07:53:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-cow-race 2024-04-27T07:53:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-cycle-write 2024-04-27T07:53:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:aio-dio-cycle-write.c: In function 'init_test': 2024-04-27T07:53:25.664 INFO:teuthology.orchestra.run.smithi022.stderr:aio-dio-cycle-write.c:83:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:25.664 INFO:teuthology.orchestra.run.smithi022.stderr: 83 | ftruncate(fd, buf_size); 2024-04-27T07:53:25.664 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-eof-race 2024-04-27T07:53:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-extend-stat 2024-04-27T07:53:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-fcntl-race 2024-04-27T07:53:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-hole-filling-race 2024-04-27T07:53:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-invalidate-failure 2024-04-27T07:53:26.661 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-invalidate-readahead 2024-04-27T07:53:26.838 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aiodio_sparse2 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c:111:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr: 111 | ftruncate(fd, filesize); 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c:274:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr: 274 | ftruncate(fd, size); 2024-04-27T07:53:26.958 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-27T07:53:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-subblock-eof-read 2024-04-27T07:53:27.251 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-write-verify 2024-04-27T07:53:27.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:27.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:27.475 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-27T07:53:27.600 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-27T07:53:27.730 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-last-ref-held-by-io 2024-04-27T07:53:27.915 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2024-04-27T07:53:27.922 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] vfstest 2024-04-27T07:53:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mount-idmapped 2024-04-27T07:53:32.722 INFO:teuthology.orchestra.run.smithi022.stdout:Building m4 2024-04-27T07:53:32.726 INFO:teuthology.orchestra.run.smithi022.stdout:Building common 2024-04-27T07:53:32.730 INFO:teuthology.orchestra.run.smithi022.stdout:Building tests 2024-04-27T07:53:32.737 INFO:teuthology.orchestra.run.smithi022.stdout:Building btrfs 2024-04-27T07:53:32.744 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/btrfs/group.list 2024-04-27T07:53:32.817 INFO:teuthology.orchestra.run.smithi022.stdout:Building ceph 2024-04-27T07:53:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ceph/group.list 2024-04-27T07:53:32.838 INFO:teuthology.orchestra.run.smithi022.stdout:Building cifs 2024-04-27T07:53:32.843 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/cifs/group.list 2024-04-27T07:53:32.856 INFO:teuthology.orchestra.run.smithi022.stdout:Building ext4 2024-04-27T07:53:32.862 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ext4/group.list 2024-04-27T07:53:32.902 INFO:teuthology.orchestra.run.smithi022.stdout:Building f2fs 2024-04-27T07:53:32.908 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/f2fs/group.list 2024-04-27T07:53:32.926 INFO:teuthology.orchestra.run.smithi022.stdout:Building generic 2024-04-27T07:53:32.934 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/generic/group.list 2024-04-27T07:53:33.034 INFO:teuthology.orchestra.run.smithi022.stdout:Building nfs 2024-04-27T07:53:33.039 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/nfs/group.list 2024-04-27T07:53:33.055 INFO:teuthology.orchestra.run.smithi022.stdout:Building ocfs2 2024-04-27T07:53:33.060 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ocfs2/group.list 2024-04-27T07:53:33.073 INFO:teuthology.orchestra.run.smithi022.stdout:Building overlay 2024-04-27T07:53:33.079 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/overlay/group.list 2024-04-27T07:53:33.123 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2024-04-27T07:53:33.128 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/perf/group.list 2024-04-27T07:53:33.141 INFO:teuthology.orchestra.run.smithi022.stdout:Building selftest 2024-04-27T07:53:33.146 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/selftest/group.list 2024-04-27T07:53:33.159 INFO:teuthology.orchestra.run.smithi022.stdout:Building shared 2024-04-27T07:53:33.165 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/shared/group.list 2024-04-27T07:53:33.183 INFO:teuthology.orchestra.run.smithi022.stdout:Building tmpfs 2024-04-27T07:53:33.189 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/tmpfs/group.list 2024-04-27T07:53:33.201 INFO:teuthology.orchestra.run.smithi022.stdout:Building udf 2024-04-27T07:53:33.206 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/udf/group.list 2024-04-27T07:53:33.219 INFO:teuthology.orchestra.run.smithi022.stdout:Building xfs 2024-04-27T07:53:33.226 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/xfs/group.list 2024-04-27T07:53:33.324 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.BsSpkJ5lk2xfstests-dev && exec sudo make install) 2024-04-27T07:53:33.347 INFO:teuthology.orchestra.run.smithi022.stdout:Building include 2024-04-27T07:53:33.351 INFO:teuthology.orchestra.run.smithi022.stdout:Building lib 2024-04-27T07:53:33.355 INFO:teuthology.orchestra.run.smithi022.stdout:Building ltp 2024-04-27T07:53:33.360 INFO:teuthology.orchestra.run.smithi022.stdout:Building src 2024-04-27T07:53:33.394 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2024-04-27T07:53:33.398 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2024-04-27T07:53:33.401 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2024-04-27T07:53:33.405 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2024-04-27T07:53:33.409 INFO:teuthology.orchestra.run.smithi022.stdout:Building m4 2024-04-27T07:53:33.412 INFO:teuthology.orchestra.run.smithi022.stdout:Building common 2024-04-27T07:53:33.416 INFO:teuthology.orchestra.run.smithi022.stdout:Building tests 2024-04-27T07:53:33.421 INFO:teuthology.orchestra.run.smithi022.stdout:Building btrfs 2024-04-27T07:53:33.427 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/btrfs/group.list 2024-04-27T07:53:33.490 INFO:teuthology.orchestra.run.smithi022.stdout:Building ceph 2024-04-27T07:53:33.495 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ceph/group.list 2024-04-27T07:53:33.510 INFO:teuthology.orchestra.run.smithi022.stdout:Building cifs 2024-04-27T07:53:33.515 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/cifs/group.list 2024-04-27T07:53:33.528 INFO:teuthology.orchestra.run.smithi022.stdout:Building ext4 2024-04-27T07:53:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ext4/group.list 2024-04-27T07:53:33.571 INFO:teuthology.orchestra.run.smithi022.stdout:Building f2fs 2024-04-27T07:53:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/f2fs/group.list 2024-04-27T07:53:33.593 INFO:teuthology.orchestra.run.smithi022.stdout:Building generic 2024-04-27T07:53:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/generic/group.list 2024-04-27T07:53:33.706 INFO:teuthology.orchestra.run.smithi022.stdout:Building nfs 2024-04-27T07:53:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/nfs/group.list 2024-04-27T07:53:33.728 INFO:teuthology.orchestra.run.smithi022.stdout:Building ocfs2 2024-04-27T07:53:33.734 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ocfs2/group.list 2024-04-27T07:53:33.750 INFO:teuthology.orchestra.run.smithi022.stdout:Building overlay 2024-04-27T07:53:33.755 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/overlay/group.list 2024-04-27T07:53:33.803 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2024-04-27T07:53:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/perf/group.list 2024-04-27T07:53:33.824 INFO:teuthology.orchestra.run.smithi022.stdout:Building selftest 2024-04-27T07:53:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/selftest/group.list 2024-04-27T07:53:33.845 INFO:teuthology.orchestra.run.smithi022.stdout:Building shared 2024-04-27T07:53:33.851 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/shared/group.list 2024-04-27T07:53:33.872 INFO:teuthology.orchestra.run.smithi022.stdout:Building tmpfs 2024-04-27T07:53:33.878 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/tmpfs/group.list 2024-04-27T07:53:33.895 INFO:teuthology.orchestra.run.smithi022.stdout:Building udf 2024-04-27T07:53:33.902 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/udf/group.list 2024-04-27T07:53:33.915 INFO:teuthology.orchestra.run.smithi022.stdout:Building xfs 2024-04-27T07:53:33.922 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/xfs/group.list 2024-04-27T07:53:34.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-27T07:53:34.035 INFO:teuthology.orchestra.run.smithi022.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-27T07:53:34.035 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-27T07:53:34.040 INFO:teuthology.orchestra.run.smithi022.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-27T07:53:34.040 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-27T07:53:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-27T07:53:34.055 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-27T07:53:34.151 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-27T07:53:34.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-27T07:53:34.185 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2024-04-27T07:53:34.190 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2024-04-27T07:53:34.195 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2024-04-27T07:53:34.201 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2024-04-27T07:53:34.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake -C log-writes install 2024-04-27T07:53:34.207 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-27T07:53:34.219 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-27T07:53:34.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake -C perf install 2024-04-27T07:53:34.233 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-27T07:53:34.245 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-27T07:53:34.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-27T07:53:34.278 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-27T07:53:34.290 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-27T07:53:34.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake -C vfs install 2024-04-27T07:53:34.355 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-27T07:53:34.366 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-27T07:53:34.383 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-27T07:53:34.393 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle t_mmap_dio uring_read_fault xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-27T07:53:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-27T07:53:36.263 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-27T07:53:36.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-27T07:53:36.298 INFO:teuthology.orchestra.run.smithi022.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-27T07:53:36.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-27T07:53:36.301 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-27T07:53:36.311 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-27T07:53:36.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-27T07:53:36.469 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/btrfs/ install 2024-04-27T07:53:36.473 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/btrfs/group.list 2024-04-27T07:53:36.545 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-27T07:53:36.557 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-27T07:53:37.672 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-27T07:53:37.687 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-27T07:53:38.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ceph/ install 2024-04-27T07:53:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ceph/group.list 2024-04-27T07:53:38.816 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-27T07:53:38.828 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-27T07:53:38.853 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-27T07:53:38.866 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-27T07:53:38.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/cifs/ install 2024-04-27T07:53:38.897 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/cifs/group.list 2024-04-27T07:53:38.913 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-27T07:53:38.925 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-27T07:53:38.938 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-27T07:53:38.950 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-27T07:53:38.962 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ext4/ install 2024-04-27T07:53:38.966 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ext4/group.list 2024-04-27T07:53:39.013 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-27T07:53:39.025 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-27T07:53:39.254 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-27T07:53:39.266 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-27T07:53:39.516 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/f2fs/ install 2024-04-27T07:53:39.521 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/f2fs/group.list 2024-04-27T07:53:39.544 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-27T07:53:39.556 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-27T07:53:39.572 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-27T07:53:39.584 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-27T07:53:39.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/generic/ install 2024-04-27T07:53:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/generic/group.list 2024-04-27T07:53:39.738 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-27T07:53:39.750 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-27T07:53:42.338 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-27T07:53:42.351 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-27T07:53:45.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/nfs/ install 2024-04-27T07:53:45.005 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/nfs/group.list 2024-04-27T07:53:45.024 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-27T07:53:45.036 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-27T07:53:45.049 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-27T07:53:45.061 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-27T07:53:45.074 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ocfs2/ install 2024-04-27T07:53:45.078 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/ocfs2/group.list 2024-04-27T07:53:45.095 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-27T07:53:45.107 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-27T07:53:45.119 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-27T07:53:45.132 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-27T07:53:45.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/overlay/ install 2024-04-27T07:53:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/overlay/group.list 2024-04-27T07:53:45.204 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-27T07:53:45.216 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-27T07:53:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-27T07:53:45.593 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-27T07:53:45.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/perf/ install 2024-04-27T07:53:45.972 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/perf/group.list 2024-04-27T07:53:45.987 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-27T07:53:46.000 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-27T07:53:46.013 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-27T07:53:46.026 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-27T07:53:46.040 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/selftest/ install 2024-04-27T07:53:46.043 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/selftest/group.list 2024-04-27T07:53:46.060 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-27T07:53:46.073 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-27T07:53:46.102 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-27T07:53:46.115 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-27T07:53:46.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/shared/ install 2024-04-27T07:53:46.149 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/shared/group.list 2024-04-27T07:53:46.169 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-27T07:53:46.180 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-27T07:53:46.200 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-27T07:53:46.211 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-27T07:53:46.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/tmpfs/ install 2024-04-27T07:53:46.234 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/tmpfs/group.list 2024-04-27T07:53:46.251 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-27T07:53:46.263 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-27T07:53:46.273 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-27T07:53:46.285 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-27T07:53:46.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/udf/ install 2024-04-27T07:53:46.300 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/udf/group.list 2024-04-27T07:53:46.313 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-27T07:53:46.324 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-27T07:53:46.336 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-27T07:53:46.347 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-27T07:53:46.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/xfs/ install 2024-04-27T07:53:46.364 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.BsSpkJ5lk2xfstests-dev/tests/xfs/group.list 2024-04-27T07:53:46.491 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-27T07:53:46.504 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-27T07:53:48.977 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-27T07:53:48.990 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-27T07:53:51.466 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-27T07:53:51.479 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-27T07:53:51.490 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-27T07:53:51.503 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-27T07:53:51.517 INFO:tasks.cephfs.test_acls:client is fuse mounted 2024-04-27T07:53:51.517 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-27T07:53:51.517 DEBUG:teuthology.orchestra.run.smithi022:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.BsSpkJ5lk2xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-27T07:53:51.755 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2024-04-27T07:53:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-27T07:53:51.760 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:52.130 INFO:teuthology.orchestra.run.smithi022.stdout:generic/099 1s 2024-04-27T07:53:52.137 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/099 2024-04-27T07:53:52.137 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2024-04-27T07:53:52.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:52.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[36960]: fuse finished with error 0 and tester_r 0 2024-04-27T07:53:52.153 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-04-27T07:53:52.153 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph-fuse PLATFORM -- Linux/x86_64 smithi022 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 generic/099 1s Ran: generic/099 Passed all 1 tests 2024-04-27T07:53:52.153 INFO:tasks.cephfs.test_acls:Command stderr - 2024-04-27T07:53:52.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo userdel --force --remove fsgqa 2024-04-27T07:53:52.220 INFO:teuthology.orchestra.run.smithi022.stderr:userdel: fsgqa mail spool (/var/mail/fsgqa) not found 2024-04-27T07:53:52.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo userdel --force --remove 123456-fsgqa 2024-04-27T07:53:52.612 INFO:teuthology.orchestra.run.smithi022.stderr:userdel: 123456-fsgqa mail spool (/var/mail/123456-fsgqa) not found 2024-04-27T07:53:52.612 INFO:teuthology.orchestra.run.smithi022.stderr:userdel: 123456-fsgqa home directory (/home/123456-fsgqa) not found 2024-04-27T07:53:52.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo groupdel fsgqa 2024-04-27T07:53:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:groupdel: group 'fsgqa' does not exist 2024-04-27T07:53:52.916 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-27T07:53:52.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:53:52.917 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.BsSpkJ5lk2xfstests-dev/results) 2024-04-27T07:53:52.963 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-27T07:53:52.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:53:52.964 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.BsSpkJ5lk2xfstests-dev 2024-04-27T07:53:52.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf /tmp/tmp.BsSpkJ5lk2xfstests-dev 2024-04-27T07:53:53.088 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-27T07:53:53.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:53:53.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:53:53.206 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-27T07:53:53.206 INFO:tasks.cephfs.mount:Terminating background process 2024-04-27T07:53:53.206 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:53:53.207 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:53:53.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21058 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-27T07:53:53.213 INFO:tasks.cephfs.mount:Terminating background process 2024-04-27T07:53:53.213 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:53:53.213 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:53:53.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16118 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-27T07:53:53.219 INFO:tasks.cephfs.mount:Terminating background process 2024-04-27T07:53:53.219 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:53:53.219 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:53:53.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21058 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-27T07:53:53.260 INFO:tasks.cephfs.mount:Terminating background process 2024-04-27T07:53:53.260 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:53:53.260 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:53:53.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16118 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-27T07:53:53.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:53:53.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 Processor -- start 2024-04-27T07:53:53.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 -- start start 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca7c144e40 0x7fca7c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c06ef10 0x7fca7c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fca7c1628e0 0x7fca7c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c145220 con 0x7fca7c144e40 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c162df0 con 0x7fca7c1628e0 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca83f6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c1630d0 con 0x7fca7c06ef10 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca7c144e40 0x7fca7c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca7c144e40 0x7fca7c06e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47330/0 (socket says 172.21.15.22:47330) 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 -- 172.21.15.22:0/1321840207 learned_addr learned my addr 172.21.15.22:0/1321840207 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81505640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c06ef10 0x7fca7c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca82507640 1 --2- 172.21.15.22:0/1321840207 >> v2:172.21.15.183:3300/0 conn(0x7fca7c1628e0 0x7fca7c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 -- 172.21.15.22:0/1321840207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c06ef10 msgr2=0x7fca7c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c06ef10 0x7fca7c1623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 -- 172.21.15.22:0/1321840207 >> v2:172.21.15.183:3300/0 conn(0x7fca7c1628e0 msgr2=0x7fca7c06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 --2- 172.21.15.22:0/1321840207 >> v2:172.21.15.183:3300/0 conn(0x7fca7c1628e0 0x7fca7c06f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.433+0000 7fca81d06640 1 -- 172.21.15.22:0/1321840207 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c06e1d0 con 0x7fca7c144e40 2024-04-27T07:53:53.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca81d06640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c144e40 0x7fca7c06e9d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca7c162300 tx=0x7fca7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a783fe64f76dff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca80d04640 1 -- 172.21.15.22:0/1321840207 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780a5780 con 0x7fca7c144e40 2024-04-27T07:53:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca80d04640 1 -- 172.21.15.22:0/1321840207 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca780a5920 con 0x7fca7c144e40 2024-04-27T07:53:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca80d04640 1 -- 172.21.15.22:0/1321840207 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780a5cf0 con 0x7fca7c144e40 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- 172.21.15.22:0/1321840207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c144e40 msgr2=0x7fca7c06e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c144e40 0x7fca7c06e9d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca7c162300 tx=0x7fca7801c9f0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- 172.21.15.22:0/1321840207 shutdown_connections 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- 172.21.15.22:0/1321840207 >> v2:172.21.15.183:3300/0 conn(0x7fca7c1628e0 0x7fca7c06f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c06ef10 0x7fca7c1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- 172.21.15.22:0/1321840207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c144e40 0x7fca7c06e9d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- 172.21.15.22:0/1321840207 >> 172.21.15.22:0/1321840207 conn(0x7fca7c067f10 msgr2=0x7fca7c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- 172.21.15.22:0/1321840207 shutdown_connections 2024-04-27T07:53:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- 172.21.15.22:0/1321840207 wait complete. 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 Processor -- start 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- start start 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c144e40 0x7fca7c0b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 0x7fca7c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c15eda0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c0af670 con 0x7fca7c1628e0 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca83f6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c15ef20 con 0x7fca7c144e40 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.437+0000 7fca82507640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47334/0 (socket says 172.21.15.22:47334) 2024-04-27T07:53:53.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 -- 172.21.15.22:0/4127699596 learned_addr learned my addr 172.21.15.22:0/4127699596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca81d06640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c144e40 0x7fca7c0b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca81505640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 0x7fca7c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 -- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c144e40 msgr2=0x7fca7c0b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c144e40 0x7fca7c0b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 -- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 msgr2=0x7fca7c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 0x7fca7c15cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 -- 172.21.15.22:0/4127699596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c140440 con 0x7fca7c13fbf0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca81505640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 0x7fca7c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca82507640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fca7400d560 tx=0x7fca7400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740125f0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74019070 con 0x7fca7c13fbf0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c140720 con 0x7fca7c13fbf0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740129f0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c15d580 con 0x7fca7c13fbf0 2024-04-27T07:53:53.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c068c30 con 0x7fca7c13fbf0 2024-04-27T07:53:53.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fca74012b90 con 0x7fca7c13fbf0 2024-04-27T07:53:53.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 0x7fca6006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fca740876c0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.441+0000 7fca81d06640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 0x7fca6006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.445+0000 7fca81d06640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 0x7fca6006a870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca780ae000 tx=0x7fca780a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.445+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca74057cf0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca7c160830 con 0x7fca7c13fbf0 2024-04-27T07:53:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca727fc640 1 -- 172.21.15.22:0/4127699596 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fca740575f0 con 0x7fca7c13fbf0 2024-04-27T07:53:53.648 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:53.648 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:53:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 msgr2=0x7fca6006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 0x7fca6006a870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca780ae000 tx=0x7fca780a7070 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 msgr2=0x7fca7c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fca7400d560 tx=0x7fca7400da20 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 shutdown_connections 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fca600683b0 0x7fca6006a870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c13fbf0 0x7fca7c15dd80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca7c1628e0 0x7fca7c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 --2- 172.21.15.22:0/4127699596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca7c144e40 0x7fca7c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 >> 172.21.15.22:0/4127699596 conn(0x7fca7c067f10 msgr2=0x7fca7c147410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 shutdown_connections 2024-04-27T07:53:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.645+0000 7fca83f6a640 1 -- 172.21.15.22:0/4127699596 wait complete. 2024-04-27T07:53:53.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:53:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.789+0000 7fa042d68640 1 Processor -- start 2024-04-27T07:53:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.789+0000 7fa042d68640 1 -- start start 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa042d68640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 0x7fa03c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa042d68640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa042d68640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03c145220 con 0x7fa03c1628e0 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa042d68640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa03c162df0 con 0x7fa03c144e40 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa042d68640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c1630d0 con 0x7fa03c06ef10 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa040b04640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 0x7fa03c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:58118/0 (socket says 172.21.15.22:58118) 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa040b04640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 0x7fa03c06e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:52248/0 (socket says 172.21.15.22:52248) 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 -- 172.21.15.22:0/3852885411 learned_addr learned my addr 172.21.15.22:0/3852885411 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa041305640 1 --1- 172.21.15.22:0/3852885411 >> v1:172.21.15.183:6789/0 conn(0x7fa03c1628e0 0x7fa03c06f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:47936/0 (socket says 172.21.15.22:47936) 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 -- 172.21.15.22:0/3852885411 >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 msgr2=0x7fa03c06e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 --2- 172.21.15.22:0/3852885411 >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 0x7fa03c06e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 -- 172.21.15.22:0/3852885411 >> v1:172.21.15.183:6789/0 conn(0x7fa03c1628e0 legacy=0x7fa03c06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.793+0000 7fa03bfff640 1 -- 172.21.15.22:0/3852885411 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c06e1d0 con 0x7fa03c06ef10 2024-04-27T07:53:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa03bfff640 1 --2- 172.21.15.22:0/3852885411 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa030009e50 tx=0x7fa03001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cafb0c5423e3ecfa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa03b7fe640 1 -- 172.21.15.22:0/3852885411 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0300a5810 con 0x7fa03c06ef10 2024-04-27T07:53:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa03b7fe640 1 -- 172.21.15.22:0/3852885411 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0300a59b0 con 0x7fa03c06ef10 2024-04-27T07:53:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa03b7fe640 1 -- 172.21.15.22:0/3852885411 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0300a5d80 con 0x7fa03c06ef10 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- 172.21.15.22:0/3852885411 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 msgr2=0x7fa03c1623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 --2- 172.21.15.22:0/3852885411 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa030009e50 tx=0x7fa03001ca30 comp rx=0 tx=0).stop 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- 172.21.15.22:0/3852885411 shutdown_connections 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 --2- 172.21.15.22:0/3852885411 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c06ef10 0x7fa03c1623a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 --2- 172.21.15.22:0/3852885411 >> v2:172.21.15.183:3300/0 conn(0x7fa03c144e40 0x7fa03c06e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- 172.21.15.22:0/3852885411 >> 172.21.15.22:0/3852885411 conn(0x7fa03c067f10 msgr2=0x7fa03c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- 172.21.15.22:0/3852885411 shutdown_connections 2024-04-27T07:53:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- 172.21.15.22:0/3852885411 wait complete. 2024-04-27T07:53:53.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 Processor -- start 2024-04-27T07:53:53.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.797+0000 7fa042d68640 1 -- start start 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 0x7fa03c0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03c13f9e0 0x7fa03c13fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0af460 con 0x7fa03c1628e0 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c06fe80 con 0x7fa03c13f9e0 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c0af5e0 con 0x7fa03c144e40 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47344/0 (socket says 172.21.15.22:47344) 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 -- 172.21.15.22:0/150239603 learned_addr learned my addr 172.21.15.22:0/150239603 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa040b04640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 0x7fa03c0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa041305640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03c13f9e0 0x7fa03c13fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 -- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 msgr2=0x7fa03c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 0x7fa03c0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 -- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03c13f9e0 msgr2=0x7fa03c13fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03c13f9e0 0x7fa03c13fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 -- 172.21.15.22:0/150239603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c0725b0 con 0x7fa03c1628e0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa040b04640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 0x7fa03c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa03bfff640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa03001a660 tx=0x7fa0300197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0300b8470 con 0x7fa03c1628e0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0300b8610 con 0x7fa03c1628e0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03c146db0 con 0x7fa03c1628e0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0300b89c0 con 0x7fa03c1628e0 2024-04-27T07:53:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03c140550 con 0x7fa03c1628e0 2024-04-27T07:53:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03c068c30 con 0x7fa03c1628e0 2024-04-27T07:53:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa0300c0020 con 0x7fa03c1628e0 2024-04-27T07:53:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 0x7fa02806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa030121360 con 0x7fa03c1628e0 2024-04-27T07:53:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.801+0000 7fa040b04640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 0x7fa02806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.805+0000 7fa040b04640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 0x7fa02806a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa034009bb0 tx=0x7fa034008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:53.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.805+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa030016620 con 0x7fa03c1628e0 2024-04-27T07:53:53.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.969+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa03c15d740 con 0x7fa03c1628e0 2024-04-27T07:53:53.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.973+0000 7fa0397fa640 1 -- 172.21.15.22:0/150239603 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7fa0300f1900 con 0x7fa03c1628e0 2024-04-27T07:53:53.977 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:53.977 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:56.149113+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:53:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 msgr2=0x7fa02806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 0x7fa02806a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa034009bb0 tx=0x7fa034008040 comp rx=0 tx=0).stop 2024-04-27T07:53:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 msgr2=0x7fa03c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa03001a660 tx=0x7fa0300197d0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 shutdown_connections 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fa028068380 0x7fa02806a840 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03c13f9e0 0x7fa03c13fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa03c1628e0 0x7fa03c15d200 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 --2- 172.21.15.22:0/150239603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa03c144e40 0x7fa03c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 >> 172.21.15.22:0/150239603 conn(0x7fa03c067f10 msgr2=0x7fa03c06f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 shutdown_connections 2024-04-27T07:53:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:53.977+0000 7fa042d68640 1 -- 172.21.15.22:0/150239603 wait complete. 2024-04-27T07:53:53.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:50:42.186099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:51:52.976189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:51:53.432250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:53:53.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 Processor -- start 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 -- start start 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 0x7f80c009b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f80c00d17a0 0x7f80c015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c00d0760 con 0x7f80c015ec70 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f80c015f050 con 0x7f80c00d17a0 2024-04-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.125+0000 7f80c65d5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00d09e0 con 0x7f80c009b140 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80bf7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 0x7f80c009b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f80c00d17a0 0x7f80c015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80bf7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 0x7f80c009b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:58138/0 (socket says 172.21.15.22:58138) 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f80c00d17a0 0x7f80c015fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:52270/0 (socket says 172.21.15.22:52270) 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80bf7fe640 1 -- 172.21.15.22:0/188376796 learned_addr learned my addr 172.21.15.22:0/188376796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 -- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 msgr2=0x7f80c009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 --2- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 0x7f80c009b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 -- 172.21.15.22:0/188376796 >> v1:172.21.15.183:6789/0 conn(0x7f80c015ec70 legacy=0x7f80c009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 -- 172.21.15.22:0/188376796 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00d0cc0 con 0x7f80c00d17a0 2024-04-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80c4b72640 1 --2- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c00d17a0 0x7f80c015fd00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009d90 tx=0x7f80b801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62cec2f85dc5278 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80beffd640 1 -- 172.21.15.22:0/188376796 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80b2070 con 0x7f80c00d17a0 2024-04-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80beffd640 1 -- 172.21.15.22:0/188376796 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b80a58b0 con 0x7f80c00d17a0 2024-04-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.129+0000 7f80beffd640 1 -- 172.21.15.22:0/188376796 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a5c80 con 0x7f80c00d17a0 2024-04-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c00d17a0 msgr2=0x7f80c015fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c00d17a0 0x7f80c015fd00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009d90 tx=0x7f80b801c8e0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/188376796 shutdown_connections 2024-04-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c00d17a0 0x7f80c015fd00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- 172.21.15.22:0/188376796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c009b140 0x7f80c009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/188376796 >> 172.21.15.22:0/188376796 conn(0x7f80c00b4080 msgr2=0x7f80c00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/188376796 shutdown_connections 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/188376796 wait complete. 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 Processor -- start 2024-04-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- start start 2024-04-27T07:53:54.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c009b140 0x7f80c00b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 0x7f80c00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80c006d880 con 0x7f80c00d17a0 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80c006d700 con 0x7f80c009b140 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00b7f50 con 0x7f80c015ec70 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47350/0 (socket says 172.21.15.22:47350) 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 -- 172.21.15.22:0/2270262883 learned_addr learned my addr 172.21.15.22:0/2270262883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bffff640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c009b140 0x7f80c00b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c4b72640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 0x7f80c00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 -- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 msgr2=0x7f80c00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 0x7f80c00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 -- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c009b140 msgr2=0x7f80c00b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c009b140 0x7f80c00b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 -- 172.21.15.22:0/2270262883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00c6be0 con 0x7f80c00d17a0 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c4b72640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 0x7f80c00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bf7fe640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e7f0 tx=0x7f80b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4017070 con 0x7f80c00d17a0 2024-04-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b400c940 con 0x7f80c00d17a0 2024-04-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b400cd30 con 0x7f80c00d17a0 2024-04-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c00c6ec0 con 0x7f80c00d17a0 2024-04-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.133+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c00c7480 con 0x7f80c00d17a0 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c004ff10 con 0x7f80c00d17a0 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f80b401a070 con 0x7f80c00d17a0 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80bcff9640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 0x7f80ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f80b4086b40 con 0x7f80c00d17a0 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80bffff640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 0x7f80ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.137+0000 7f80bffff640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 0x7f80ac06a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80b0001810 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.141+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80b4056ab0 con 0x7f80c00d17a0 2024-04-27T07:53:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.305+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80c00b5050 con 0x7f80c00d17a0 2024-04-27T07:53:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.305+0000 7f80bcff9640 1 -- 172.21.15.22:0/2270262883 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f80b405adb0 con 0x7f80c00d17a0 2024-04-27T07:53:54.310 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-27T07:53:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 msgr2=0x7f80ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 0x7f80ac06a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80b0001810 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2024-04-27T07:53:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 msgr2=0x7f80c0151050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e7f0 tx=0x7f80b400ecb0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 shutdown_connections 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f80ac068380 0x7f80ac06a840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80c015ec70 0x7f80c00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c00d17a0 0x7f80c0151050 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 --2- 172.21.15.22:0/2270262883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c009b140 0x7f80c00b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 >> 172.21.15.22:0/2270262883 conn(0x7f80c00b4080 msgr2=0x7f80c0154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 shutdown_connections 2024-04-27T07:53:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.309+0000 7f80c65d5640 1 -- 172.21.15.22:0/2270262883 wait complete. 2024-04-27T07:53:54.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:53:54.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 Processor -- start 2024-04-27T07:53:54.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 -- start start 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b100cfed0 0x7f5b1015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1015f0b0 0x7f5b100ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b100d0360 con 0x7f5b1015ecd0 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5b100b2090 con 0x7f5b100cfed0 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b165b7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100b2370 con 0x7f5b1015f0b0 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b100cfed0 0x7f5b1015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5b1015ecd0 0x7f5b1015f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44156/0 (socket says 172.21.15.22:44156) 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0ffff640 1 -- 172.21.15.22:0/3236468749 learned_addr learned my addr 172.21.15.22:0/3236468749 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b14b54640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1015f0b0 0x7f5b100ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0f7fe640 1 -- 172.21.15.22:0/3236468749 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1015f0b0 msgr2=0x7f5b100ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1015f0b0 0x7f5b100ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0f7fe640 1 -- 172.21.15.22:0/3236468749 >> v1:172.21.15.22:6789/0 conn(0x7f5b1015ecd0 legacy=0x7f5b1015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0effd640 1 -- 172.21.15.22:0/3236468749 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915434318 0 0) 0x7f5b100d0360 con 0x7f5b1015ecd0 2024-04-27T07:53:54.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.453+0000 7f5b0f7fe640 1 -- 172.21.15.22:0/3236468749 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b100b2650 con 0x7f5b100cfed0 2024-04-27T07:53:54.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b100cfed0 0x7f5b1015fc70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5af80097f0 tx=0x7f5af801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cfc2115460fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0effd640 1 -- 172.21.15.22:0/3236468749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af80ab600 con 0x7f5b100cfed0 2024-04-27T07:53:54.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0effd640 1 -- 172.21.15.22:0/3236468749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af80b0070 con 0x7f5b100cfed0 2024-04-27T07:53:54.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0effd640 1 -- 172.21.15.22:0/3236468749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af80ab9c0 con 0x7f5b100cfed0 2024-04-27T07:53:54.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- 172.21.15.22:0/3236468749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b100cfed0 msgr2=0x7f5b1015fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b100cfed0 0x7f5b1015fc70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5af80097f0 tx=0x7f5af801c470 comp rx=0 tx=0).stop 2024-04-27T07:53:54.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- 172.21.15.22:0/3236468749 shutdown_connections 2024-04-27T07:53:54.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1015f0b0 0x7f5b100ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- 172.21.15.22:0/3236468749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b100cfed0 0x7f5b1015fc70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- 172.21.15.22:0/3236468749 >> 172.21.15.22:0/3236468749 conn(0x7f5b1009aa30 msgr2=0x7f5b100b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- 172.21.15.22:0/3236468749 shutdown_connections 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- 172.21.15.22:0/3236468749 wait complete. 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 Processor -- start 2024-04-27T07:53:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- start start 2024-04-27T07:53:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 0x7f5b1015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 0x7f5b100c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100af260 con 0x7f5b1015ecd0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100af0e0 con 0x7f5b1015f0b0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b165b7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100af3e0 con 0x7f5b1014b910 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47366/0 (socket says 172.21.15.22:47366) 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.457+0000 7f5b0ffff640 1 -- 172.21.15.22:0/705811711 learned_addr learned my addr 172.21.15.22:0/705811711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 0x7f5b1015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b14b54640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 0x7f5b100c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 -- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 msgr2=0x7f5b100c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 0x7f5b100c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 -- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 msgr2=0x7f5b1015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 0x7f5b1015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 -- 172.21.15.22:0/705811711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1014bda0 con 0x7f5b1015ecd0 2024-04-27T07:53:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 0x7f5b1015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b14b54640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 0x7f5b100c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0ffff640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c890 tx=0x7f5b0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00018070 con 0x7f5b1015ecd0 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0000a640 con 0x7f5b1015ecd0 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b100cb610 con 0x7f5b1015ecd0 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000aa30 con 0x7f5b1015ecd0 2024-04-27T07:53:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1015a480 con 0x7f5b1015ecd0 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1004ff10 con 0x7f5b1015ecd0 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b0001b070 con 0x7f5b1015ecd0 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 0x7f5af006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5b00085fd0 con 0x7f5b1015ecd0 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 0x7f5af006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.461+0000 7f5b0f7fe640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 0x7f5af006a870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5af80af000 tx=0x7f5af8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.465+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b00056620 con 0x7f5b1015ecd0 2024-04-27T07:53:54.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.629+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b10073380 con 0x7f5b1015ecd0 2024-04-27T07:53:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.629+0000 7f5b0cff9640 1 -- 172.21.15.22:0/705811711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f5b00055f20 con 0x7f5b1015ecd0 2024-04-27T07:53:54.633 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-27T07:53:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 msgr2=0x7f5af006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 0x7f5af006a870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5af80af000 tx=0x7f5af8008040 comp rx=0 tx=0).stop 2024-04-27T07:53:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 msgr2=0x7f5b100c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c890 tx=0x7f5b0000cd50 comp rx=0 tx=0).stop 2024-04-27T07:53:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 shutdown_connections 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5af00683b0 0x7f5af006a870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b1014b910 0x7f5b100c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1015f0b0 0x7f5b1015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 --2- 172.21.15.22:0/705811711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b1015ecd0 0x7f5b100c87e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 >> 172.21.15.22:0/705811711 conn(0x7f5b1009aa30 msgr2=0x7f5b1006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 shutdown_connections 2024-04-27T07:53:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.633+0000 7f5b165b7640 1 -- 172.21.15.22:0/705811711 wait complete. 2024-04-27T07:53:54.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:53:54.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 Processor -- start 2024-04-27T07:53:54.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 -- start start 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f446415ecd0 0x7f4464072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f44640b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44640b5390 con 0x7f44640b3a10 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f44640b5670 con 0x7f446415ecd0 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446bc5e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f44640ce340 con 0x7f44640b4ee0 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f44640b3a10 0x7f44640cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44158/0 (socket says 172.21.15.22:44158) 2024-04-27T07:53:54.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 -- 172.21.15.22:0/2733665930 learned_addr learned my addr 172.21.15.22:0/2733665930 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44691f9640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f44640b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44699fa640 1 --2- 172.21.15.22:0/2733665930 >> v2:172.21.15.183:3300/0 conn(0x7f446415ecd0 0x7f4464072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753937146 0 0) 0x7f44640b5390 con 0x7f44640b3a10 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4450003530 con 0x7f44640b3a10 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2967472711 0 0) 0x7f4450003530 con 0x7f44640b3a10 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 msgr2=0x7f44640b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f44640b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 >> v2:172.21.15.183:3300/0 conn(0x7f446415ecd0 msgr2=0x7f4464072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 --2- 172.21.15.22:0/2733665930 >> v2:172.21.15.183:3300/0 conn(0x7f446415ecd0 0x7f4464072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44640ce620 con 0x7f44640b3a10 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f445c001680 con 0x7f44640b3a10 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 >> v1:172.21.15.22:6789/0 conn(0x7f44640b3a10 legacy=0x7f44640cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44500039c0 0x7f4450003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f4450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450004d50 0x7f44500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44640b5670 con 0x7f4450004d50 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44640ce340 con 0x7f44500039c0 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f445000dbe0 con 0x7f44640b4ee0 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450004d50 0x7f44500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44699fa640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44500039c0 0x7f4450003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f44691f9640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f4450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 -- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 msgr2=0x7f4450004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44640b4ee0 0x7f4450004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 -- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44500039c0 msgr2=0x7f4450003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44500039c0 0x7f4450003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.781+0000 7f446a1fb640 1 -- 172.21.15.22:0/2733665930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4450003530 con 0x7f4450004d50 2024-04-27T07:53:54.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446a1fb640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450004d50 0x7f44500051d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c1c0 tx=0x7f445c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e24e6f978ddc760 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c007d00 con 0x7f4450004d50 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c0a9020 con 0x7f4450004d50 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f44689f8640 1 -- 172.21.15.22:0/2733665930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a5b80 con 0x7f4450004d50 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- 172.21.15.22:0/2733665930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450004d50 msgr2=0x7f44500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 --2- 172.21.15.22:0/2733665930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450004d50 0x7f44500051d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c1c0 tx=0x7f445c01cc60 comp rx=0 tx=0).stop 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446a1fb640 1 -- 172.21.15.22:0/2733665930 reap_dead start 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- 172.21.15.22:0/2733665930 shutdown_connections 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- 172.21.15.22:0/2733665930 >> 172.21.15.22:0/2733665930 conn(0x7f44640b21d0 msgr2=0x7f44640b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- 172.21.15.22:0/2733665930 shutdown_connections 2024-04-27T07:53:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- 172.21.15.22:0/2733665930 wait complete. 2024-04-27T07:53:54.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 Processor -- start 2024-04-27T07:53:54.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.785+0000 7f446bc5e640 1 -- start start 2024-04-27T07:53:54.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4450004d50 0x7f4464141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4464154f40 0x7f44641553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4464073040 con 0x7f4464158e60 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4464072ec0 con 0x7f4450004d50 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f44640cf530 con 0x7f4464154f40 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47376/0 (socket says 172.21.15.22:47376) 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 -- 172.21.15.22:0/596148379 learned_addr learned my addr 172.21.15.22:0/596148379 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446a1fb640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4464154f40 0x7f44641553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44699fa640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4450004d50 0x7f4464141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 -- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4464154f40 msgr2=0x7f44641553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4464154f40 0x7f44641553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 -- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4450004d50 msgr2=0x7f4464141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4450004d50 0x7f4464141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 -- 172.21.15.22:0/596148379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44641f0610 con 0x7f4464158e60 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44691f9640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4454011590 tx=0x7f4454011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004220 con 0x7f4464158e60 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44540043c0 con 0x7f4464158e60 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4464155ce0 con 0x7f4464158e60 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454015870 con 0x7f4464158e60 2024-04-27T07:53:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4464156610 con 0x7f4464158e60 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4464156200 con 0x7f4464158e60 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4454015a10 con 0x7f4464158e60 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 0x7f444806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4454087080 con 0x7f4464158e60 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.789+0000 7f44699fa640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 0x7f444806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.793+0000 7f44699fa640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 0x7f444806a870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f44640cd1b0 tx=0x7f446000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.797+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4464156200 con 0x7f4464158e60 2024-04-27T07:53:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.957+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f446404ff10 con 0x7f4464158e60 2024-04-27T07:53:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.957+0000 7f445a7fc640 1 -- 172.21.15.22:0/596148379 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f4454019030 con 0x7f4464158e60 2024-04-27T07:53:54.962 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 msgr2=0x7f444806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 0x7f444806a870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f44640cd1b0 tx=0x7f446000b040 comp rx=0 tx=0).stop 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 msgr2=0x7f44641592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4454011590 tx=0x7f4454011a50 comp rx=0 tx=0).stop 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 shutdown_connections 2024-04-27T07:53:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f44480683b0 0x7f444806a870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4464154f40 0x7f44641553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4464158e60 0x7f44641592e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 --2- 172.21.15.22:0/596148379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4450004d50 0x7f4464141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 >> 172.21.15.22:0/596148379 conn(0x7f44640b21d0 msgr2=0x7f44640b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 shutdown_connections 2024-04-27T07:53:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:54.961+0000 7f446bc5e640 1 -- 172.21.15.22:0/596148379 wait complete. 2024-04-27T07:53:54.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:53:55.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.105+0000 7f0205d7b640 1 Processor -- start 2024-04-27T07:53:55.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.105+0000 7f0205d7b640 1 -- start start 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f0205d7b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0200144dd0 0x7f020006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f0205d7b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f02000cfe10 0x7f02000b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f0205d7b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02000720d0 con 0x7f020006f6a0 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f0205d7b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f020006fb00 con 0x7f0200144dd0 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f0205d7b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0200072370 con 0x7f02000cfe10 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0200144dd0 0x7f020006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01feffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f020006f6a0 0x7f02000cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44160/0 (socket says 172.21.15.22:44160) 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01feffd640 1 -- 172.21.15.22:0/2137346290 learned_addr learned my addr 172.21.15.22:0/2137346290 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01fffff640 1 --2- 172.21.15.22:0/2137346290 >> v2:172.21.15.183:3300/0 conn(0x7f02000cfe10 0x7f02000b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 -- 172.21.15.22:0/2137346290 >> v2:172.21.15.183:3300/0 conn(0x7f02000cfe10 msgr2=0x7f02000b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:55.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2137346290 >> v2:172.21.15.183:3300/0 conn(0x7f02000cfe10 0x7f02000b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 -- 172.21.15.22:0/2137346290 >> v1:172.21.15.22:6789/0 conn(0x7f020006f6a0 legacy=0x7f02000cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 -- 172.21.15.22:0/2137346290 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0200072650 con 0x7f0200144dd0 2024-04-27T07:53:55.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01fe7fc640 1 -- 172.21.15.22:0/2137346290 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606361807 0 0) 0x7f02000720d0 con 0x7f020006f6a0 2024-04-27T07:53:55.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2137346290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020006f160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f02000b5590 tx=0x7f01f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2231a17dc7fc8d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01fe7fc640 1 -- 172.21.15.22:0/2137346290 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f00b0070 con 0x7f0200144dd0 2024-04-27T07:53:55.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01fe7fc640 1 -- 172.21.15.22:0/2137346290 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f00ab600 con 0x7f0200144dd0 2024-04-27T07:53:55.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.109+0000 7f01fe7fc640 1 -- 172.21.15.22:0/2137346290 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f00ab9d0 con 0x7f0200144dd0 2024-04-27T07:53:55.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2137346290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 msgr2=0x7f020006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2137346290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020006f160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f02000b5590 tx=0x7f01f001c480 comp rx=0 tx=0).stop 2024-04-27T07:53:55.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2137346290 shutdown_connections 2024-04-27T07:53:55.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2137346290 >> v2:172.21.15.183:3300/0 conn(0x7f02000cfe10 0x7f02000b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2137346290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020006f160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2137346290 >> 172.21.15.22:0/2137346290 conn(0x7f0200067f10 msgr2=0x7f020006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2137346290 shutdown_connections 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2137346290 wait complete. 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 Processor -- start 2024-04-27T07:53:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- start start 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 0x7f0200158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02000cfe10 0x7f020013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f02000b4ca0 con 0x7f0200144dd0 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02000b4b20 con 0x7f020006f6a0 2024-04-27T07:53:55.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f02000b4e20 con 0x7f02000cfe10 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47398/0 (socket says 172.21.15.22:47398) 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 -- 172.21.15.22:0/2123381830 learned_addr learned my addr 172.21.15.22:0/2123381830 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 0x7f0200158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01feffd640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02000cfe10 0x7f020013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 -- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02000cfe10 msgr2=0x7f020013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02000cfe10 0x7f020013f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 -- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 msgr2=0x7f0200158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 0x7f0200158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 -- 172.21.15.22:0/2123381830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0200136740 con 0x7f0200144dd0 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 0x7f0200158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01fffff640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007e90 tx=0x7f01f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8014420 con 0x7f0200144dd0 2024-04-27T07:53:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f8019070 con 0x7f0200144dd0 2024-04-27T07:53:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02001352c0 con 0x7f0200144dd0 2024-04-27T07:53:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8014800 con 0x7f0200144dd0 2024-04-27T07:53:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.113+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0200140a30 con 0x7f0200144dd0 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0200068c30 con 0x7f0200144dd0 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01f80149a0 con 0x7f0200144dd0 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f01f7fff640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 0x7f01e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f01f80871b0 con 0x7f0200144dd0 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 0x7f01e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.117+0000 7f01ff7fe640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 0x7f01e006a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01f00ad000 tx=0x7f01f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.121+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0200136740 con 0x7f0200144dd0 2024-04-27T07:53:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02000b3670 con 0x7f0200144dd0 2024-04-27T07:53:55.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f01f7fff640 1 -- 172.21.15.22:0/2123381830 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f01f8011260 con 0x7f0200144dd0 2024-04-27T07:53:55.328 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:55.328 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:53:55.328 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:53:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 msgr2=0x7f01e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 0x7f01e006a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01f00ad000 tx=0x7f01f0008040 comp rx=0 tx=0).stop 2024-04-27T07:53:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 msgr2=0x7f020013c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007e90 tx=0x7f01f800e4b0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 shutdown_connections 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f01e0068380 0x7f01e006a840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0200144dd0 0x7f020013c6b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02000cfe10 0x7f020013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 --2- 172.21.15.22:0/2123381830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f020006f6a0 0x7f0200158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 >> 172.21.15.22:0/2123381830 conn(0x7f0200067f10 msgr2=0x7f020015a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 shutdown_connections 2024-04-27T07:53:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.325+0000 7f0205d7b640 1 -- 172.21.15.22:0/2123381830 wait complete. 2024-04-27T07:53:55.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:53:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 Processor -- start 2024-04-27T07:53:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 -- start start 2024-04-27T07:53:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 0x7f71401677e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714015ce60 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714015d120 con 0x7f71400b72c0 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f7145bcd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f714015d400 con 0x7f714006c2e0 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f714015fd50 0x7f7140160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44176/0 (socket says 172.21.15.22:44176) 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713f7fe640 1 -- 172.21.15.22:0/526411197 learned_addr learned my addr 172.21.15.22:0/526411197 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713ffff640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 0x7f71401677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285188979 0 0) 0x7f714015ce60 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7128003530 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 602579830 0 0) 0x7f7128003530 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 msgr2=0x7f71401677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 0x7f71401677e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 >> v1:172.21.15.183:6789/0 conn(0x7f71400b72c0 legacy=0x7f714006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714015d6e0 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f7134001680 con 0x7f714015fd50 2024-04-27T07:53:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 >> v1:172.21.15.22:6789/0 conn(0x7f714015fd50 legacy=0x7f7140160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 0x7f71280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7128003f70 0x7f7128004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7128004d50 0x7f71280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f714015d400 con 0x7f7128003f70 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f712800bab0 con 0x7f7128004d50 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f712800bcb0 con 0x7f714006c2e0 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7128003f70 0x7f7128004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713ffff640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7128004d50 0x7f71280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 -- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 msgr2=0x7f71280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f714006c2e0 0x7f71280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 -- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7128004d50 msgr2=0x7f71280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7128004d50 0x7f71280051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.469+0000 7f713effd640 1 -- 172.21.15.22:0/526411197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7128003530 con 0x7f7128003f70 2024-04-27T07:53:55.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7128003f70 0x7f7128004810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7130001780 tx=0x7f713001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b935d1c893eb08a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71300013a0 con 0x7f7128003f70 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71300a9070 con 0x7f7128003f70 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713e7fc640 1 -- 172.21.15.22:0/526411197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130003600 con 0x7f7128003f70 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- 172.21.15.22:0/526411197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7128003f70 msgr2=0x7f7128004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 --2- 172.21.15.22:0/526411197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7128003f70 0x7f7128004810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7130001780 tx=0x7f713001c420 comp rx=0 tx=0).stop 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713ffff640 1 -- 172.21.15.22:0/526411197 reap_dead start 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- 172.21.15.22:0/526411197 shutdown_connections 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- 172.21.15.22:0/526411197 >> 172.21.15.22:0/526411197 conn(0x7f714009aa30 msgr2=0x7f71400b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- 172.21.15.22:0/526411197 shutdown_connections 2024-04-27T07:53:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- 172.21.15.22:0/526411197 wait complete. 2024-04-27T07:53:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 Processor -- start 2024-04-27T07:53:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- start start 2024-04-27T07:53:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7128003f70 0x7f71400cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 0x7f71400ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f714015f1e0 con 0x7f71400d00a0 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7140160130 con 0x7f71400cdf20 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f7145bcd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f714015f360 con 0x7f7128003f70 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47404/0 (socket says 172.21.15.22:47404) 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 -- 172.21.15.22:0/2873830269 learned_addr learned my addr 172.21.15.22:0/2873830269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713f7fe640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7128003f70 0x7f71400cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713ffff640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 0x7f71400ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 -- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7128003f70 msgr2=0x7f71400cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.473+0000 7f713effd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7128003f70 0x7f71400cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713effd640 1 -- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 msgr2=0x7f71400ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713effd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 0x7f71400ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713effd640 1 -- 172.21.15.22:0/2873830269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7140146590 con 0x7f71400d00a0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713ffff640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 0x7f71400ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713effd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7130005040 tx=0x7f7128005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71300b2040 con 0x7f71400d00a0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71300bc070 con 0x7f71400d00a0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7140146870 con 0x7f71400d00a0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130007a40 con 0x7f71400d00a0 2024-04-27T07:53:55.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71400b4a10 con 0x7f71400d00a0 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71400b4520 con 0x7f71400d00a0 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f71300ab070 con 0x7f71400d00a0 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 0x7f712c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7130120580 con 0x7f71400d00a0 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713f7fe640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 0x7f712c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.477+0000 7f713f7fe640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 0x7f712c06a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71340011f0 tx=0x7f7134001480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.481+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71300c6020 con 0x7f71400d00a0 2024-04-27T07:53:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.645+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7140069340 con 0x7f71400d00a0 2024-04-27T07:53:55.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7127fff640 1 -- 172.21.15.22:0/2873830269 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7f7130016130 con 0x7f71400d00a0 2024-04-27T07:53:55.651 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:55.652 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:56.149113+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:53:55.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 msgr2=0x7f712c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 0x7f712c06a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71340011f0 tx=0x7f7134001480 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 msgr2=0x7f71400cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7130005040 tx=0x7f7128005840 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 shutdown_connections 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f712c068380 0x7f712c06a840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71400cdf20 0x7f71400ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71400d00a0 0x7f71400cd9e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 --2- 172.21.15.22:0/2873830269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7128003f70 0x7f71400cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 >> 172.21.15.22:0/2873830269 conn(0x7f714009aa30 msgr2=0x7f71400705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 shutdown_connections 2024-04-27T07:53:55.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.649+0000 7f7145bcd640 1 -- 172.21.15.22:0/2873830269 wait complete. 2024-04-27T07:53:55.672 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:53:55.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:53:55.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.793+0000 7fccaeafd640 1 Processor -- start 2024-04-27T07:53:55.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.793+0000 7fccaeafd640 1 -- start start 2024-04-27T07:53:55.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 0x7fcca80cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcca80cfc40 0x7fcca80b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca80720d0 con 0x7fcca8144dd0 2024-04-27T07:53:55.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcca8072370 con 0x7fcca80cfc40 2024-04-27T07:53:55.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fcca8072650 con 0x7fcca806f870 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcca80cfc40 0x7fcca80b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccac899640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcca8144dd0 0x7fcca806f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44186/0 (socket says 172.21.15.22:44186) 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccac899640 1 -- 172.21.15.22:0/3412211001 learned_addr learned my addr 172.21.15.22:0/3412211001 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca7fff640 1 --2- 172.21.15.22:0/3412211001 >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 0x7fcca80cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 -- 172.21.15.22:0/3412211001 >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 msgr2=0x7fcca80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 --2- 172.21.15.22:0/3412211001 >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 0x7fcca80cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 -- 172.21.15.22:0/3412211001 >> v1:172.21.15.22:6789/0 conn(0x7fcca8144dd0 legacy=0x7fcca806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 -- 172.21.15.22:0/3412211001 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca8072930 con 0x7fcca80cfc40 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca77fe640 1 -- 172.21.15.22:0/3412211001 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164004522 0 0) 0x7fcca80720d0 con 0x7fcca8144dd0 2024-04-27T07:53:55.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca7fff640 1 --2- 172.21.15.22:0/3412211001 >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 0x7fcca80cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:55.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccad09a640 1 --2- 172.21.15.22:0/3412211001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca80cfc40 0x7fcca80b3dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcca80703f0 tx=0x7fcca001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f64d62dacd72b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca77fe640 1 -- 172.21.15.22:0/3412211001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b2070 con 0x7fcca80cfc40 2024-04-27T07:53:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca77fe640 1 -- 172.21.15.22:0/3412211001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca00a58c0 con 0x7fcca80cfc40 2024-04-27T07:53:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fcca77fe640 1 -- 172.21.15.22:0/3412211001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00a5c90 con 0x7fcca80cfc40 2024-04-27T07:53:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- 172.21.15.22:0/3412211001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca80cfc40 msgr2=0x7fcca80b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3412211001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca80cfc40 0x7fcca80b3dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcca80703f0 tx=0x7fcca001c930 comp rx=0 tx=0).stop 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- 172.21.15.22:0/3412211001 shutdown_connections 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3412211001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca80cfc40 0x7fcca80b3dd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3412211001 >> v2:172.21.15.183:3300/0 conn(0x7fcca806f870 0x7fcca80cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- 172.21.15.22:0/3412211001 >> 172.21.15.22:0/3412211001 conn(0x7fcca8067f10 msgr2=0x7fcca806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- 172.21.15.22:0/3412211001 shutdown_connections 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.797+0000 7fccaeafd640 1 -- 172.21.15.22:0/3412211001 wait complete. 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 Processor -- start 2024-04-27T07:53:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- start start 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcca80cfc40 0x7fcca80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 0x7fcca815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcca80b4ca0 con 0x7fcca806f870 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcca80b4b20 con 0x7fcca8144dd0 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcca80b4e20 con 0x7fcca80cfc40 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47418/0 (socket says 172.21.15.22:47418) 2024-04-27T07:53:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 -- 172.21.15.22:0/3690142896 learned_addr learned my addr 172.21.15.22:0/3690142896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fcca7fff640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcca80cfc40 0x7fcca80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 -- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcca80cfc40 msgr2=0x7fcca80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcca80cfc40 0x7fcca80ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 -- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 msgr2=0x7fcca815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccad09a640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 0x7fcca815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 0x7fcca815b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 -- 172.21.15.22:0/3690142896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca8140f40 con 0x7fcca806f870 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccad09a640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 0x7fcca815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccac899640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c480 tx=0x7fcc9800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98018070 con 0x7fcca806f870 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc98007500 con 0x7fcca806f870 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca8141220 con 0x7fcca806f870 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98004a30 con 0x7fcca806f870 2024-04-27T07:53:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81417e0 con 0x7fcca806f870 2024-04-27T07:53:55.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.801+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca8068c30 con 0x7fcca806f870 2024-04-27T07:53:55.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.805+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcc9801b070 con 0x7fcca806f870 2024-04-27T07:53:55.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.805+0000 7fcca57fa640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 0x7fcc9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.805+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcc98085d60 con 0x7fcca806f870 2024-04-27T07:53:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.805+0000 7fcca7fff640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 0x7fcc9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.805+0000 7fcca7fff640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 0x7fcc9406a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:55.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:55.809+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcc98056310 con 0x7fcca806f870 2024-04-27T07:53:56.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.005+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcca813e5a0 con 0x7fcca806f870 2024-04-27T07:53:56.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fcca57fa640 1 -- 172.21.15.22:0/3690142896 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fcc98055c10 con 0x7fcca806f870 2024-04-27T07:53:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:56.012 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:53:56.012 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 msgr2=0x7fcc9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 0x7fcc9406a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c008040 comp rx=0 tx=0).stop 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 msgr2=0x7fcca80c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c480 tx=0x7fcc9800c940 comp rx=0 tx=0).stop 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 shutdown_connections 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fcc94068380 0x7fcc9406a840 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcca8144dd0 0x7fcca815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcca80cfc40 0x7fcca80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 --2- 172.21.15.22:0/3690142896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcca806f870 0x7fcca80c5700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 >> 172.21.15.22:0/3690142896 conn(0x7fcca8067f10 msgr2=0x7fcca80b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 shutdown_connections 2024-04-27T07:53:56.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.009+0000 7fccaeafd640 1 -- 172.21.15.22:0/3690142896 wait complete. 2024-04-27T07:53:56.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:53:56.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 Processor -- start 2024-04-27T07:53:56.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 -- start start 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f16340cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340b3d30 con 0x7f16340d4790 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340ba7a0 con 0x7f16340b7930 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1638b62640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f16340d4540 con 0x7f163415ebe0 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163259c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f16340d4790 0x7f16340d4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44192/0 (socket says 172.21.15.22:44192) 2024-04-27T07:53:56.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163259c640 1 -- 172.21.15.22:0/1122088016 learned_addr learned my addr 172.21.15.22:0/1122088016 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f16340cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658660092 0 0) 0x7f1624001120 con 0x7f16340d4790 2024-04-27T07:53:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f161c003530 con 0x7f16340d4790 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288462015 0 0) 0x7f16340ba7a0 con 0x7f16340b7930 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1624001120 con 0x7f16340b7930 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3021197179 0 0) 0x7f161c003530 con 0x7f16340d4790 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 msgr2=0x7f16340cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f16340cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 >> v1:172.21.15.183:6789/0 conn(0x7f16340b7930 legacy=0x7f16340d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16340ba4c0 con 0x7f16340d4790 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f1624001850 con 0x7f16340d4790 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 >> v1:172.21.15.22:6789/0 conn(0x7f16340d4790 legacy=0x7f16340d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f161c003b80 0x7f161c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f161c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f161c0051d0 0x7f161c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16340d4540 con 0x7f161c004810 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f161c00bf10 con 0x7f161c003b80 2024-04-27T07:53:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f161c00c0c0 con 0x7f161c0051d0 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f161c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163259c640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f161c003b80 0x7f161c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 -- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f161c0051d0 msgr2=0x7f161c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1632d9d640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f161c0051d0 0x7f161c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f161c0051d0 0x7f161c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 -- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f161c003b80 msgr2=0x7f161c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f161c003b80 0x7f161c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1631d9b640 1 -- 172.21.15.22:0/1122088016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c003530 con 0x7f161c004810 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f1632d9d640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f161c0051d0 0x7f161c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.145+0000 7f163259c640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f161c003b80 0x7f161c003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1631d9b640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f161c004c90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1628009ab0 tx=0x7f162801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08aa016d8007bde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280b2070 con 0x7f161c004810 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16280a55f0 con 0x7f161c004810 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163159a640 1 -- 172.21.15.22:0/1122088016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280096b0 con 0x7f161c004810 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- 172.21.15.22:0/1122088016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 msgr2=0x7f161c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 --2- 172.21.15.22:0/1122088016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f161c004c90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1628009ab0 tx=0x7f162801c9c0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1632d9d640 1 -- 172.21.15.22:0/1122088016 reap_dead start 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- 172.21.15.22:0/1122088016 shutdown_connections 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- 172.21.15.22:0/1122088016 >> 172.21.15.22:0/1122088016 conn(0x7f1634067d10 msgr2=0x7f1634068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- 172.21.15.22:0/1122088016 shutdown_connections 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- 172.21.15.22:0/1122088016 wait complete. 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 Processor -- start 2024-04-27T07:53:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- start start 2024-04-27T07:53:56.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163415c560 0x7f163415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f163415f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16340d3040 con 0x7f161c004810 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16340d2ec0 con 0x7f163415c560 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f1638b62640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f16340d31c0 con 0x7f163415ebe0 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47442/0 (socket says 172.21.15.22:47442) 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.149+0000 7f163259c640 1 -- 172.21.15.22:0/606430250 learned_addr learned my addr 172.21.15.22:0/606430250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1632d9d640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f163415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1631d9b640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163415c560 0x7f163415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 -- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 msgr2=0x7f163415f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f163415f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 -- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163415c560 msgr2=0x7f163415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163415c560 0x7f163415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 -- 172.21.15.22:0/606430250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163415b7a0 con 0x7f161c004810 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f163259c640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f162400dae0 tx=0x7f1624007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624019070 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16240074e0 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163415ba80 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240047f0 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163415c040 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1634068c30 con 0x7f161c004810 2024-04-27T07:53:56.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1624004990 con 0x7f161c004810 2024-04-27T07:53:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 0x7f161406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f16240865c0 con 0x7f161c004810 2024-04-27T07:53:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1631d9b640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 0x7f161406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.153+0000 7f1631d9b640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 0x7f161406a870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16280af000 tx=0x7f16280046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.157+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16340b3d30 con 0x7f161c004810 2024-04-27T07:53:56.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16340cb590 con 0x7f161c004810 2024-04-27T07:53:56.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1622ffd640 1 -- 172.21.15.22:0/606430250 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f1624056ae0 con 0x7f161c004810 2024-04-27T07:53:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:56.356 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:51:54.480954+0000","modified":"2024-04-27T07:51:57.571057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/3341477719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3341477719},{"type":"v1","addr":"172.21.15.22:6834","nonce":3341477719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-04-27T07:53:56.356 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-04-27T07:53:56.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 msgr2=0x7f161406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 0x7f161406a870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16280af000 tx=0x7f16280046d0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 msgr2=0x7f16340ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f162400dae0 tx=0x7f1624007e40 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 shutdown_connections 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f16140683b0 0x7f161406a870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f163415ebe0 0x7f163415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163415c560 0x7f163415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 --2- 172.21.15.22:0/606430250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161c004810 0x7f16340ca760 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 >> 172.21.15.22:0/606430250 conn(0x7f1634067d10 msgr2=0x7f163415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 shutdown_connections 2024-04-27T07:53:56.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.353+0000 7f1638b62640 1 -- 172.21.15.22:0/606430250 wait complete. 2024-04-27T07:53:56.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:53:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.493+0000 7f5fb2212640 1 Processor -- start 2024-04-27T07:53:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.493+0000 7f5fb2212640 1 -- start start 2024-04-27T07:53:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fb2212640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 0x7f5fac072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fb2212640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fb2212640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac06e050 con 0x7f5fac1454b0 2024-04-27T07:53:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fb2212640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5fac06f330 con 0x7f5fac072740 2024-04-27T07:53:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fb2212640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac0ceae0 con 0x7f5fac145a00 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5faaffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 0x7f5fac072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:58234/0 (socket says 172.21.15.22:58234) 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 -- 172.21.15.22:0/2141426359 learned_addr learned my addr 172.21.15.22:0/2141426359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 -- 172.21.15.22:0/2141426359 >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 msgr2=0x7f5fac072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 --2- 172.21.15.22:0/2141426359 >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 0x7f5fac072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 -- 172.21.15.22:0/2141426359 >> v1:172.21.15.183:6789/0 conn(0x7f5fac1454b0 legacy=0x7f5fac072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 -- 172.21.15.22:0/2141426359 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0cedc0 con 0x7f5fac145a00 2024-04-27T07:53:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5faaffd640 1 --2- 172.21.15.22:0/2141426359 >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 0x7f5fac072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:53:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5fabfff640 1 --2- 172.21.15.22:0/2141426359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009e50 tx=0x7f5fa401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760ec619afa652dc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.497+0000 7f5faa7fc640 1 -- 172.21.15.22:0/2141426359 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40a5700 con 0x7f5fac145a00 2024-04-27T07:53:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faa7fc640 1 -- 172.21.15.22:0/2141426359 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa40a58a0 con 0x7f5fac145a00 2024-04-27T07:53:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faa7fc640 1 -- 172.21.15.22:0/2141426359 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40a5c70 con 0x7f5fac145a00 2024-04-27T07:53:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/2141426359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 msgr2=0x7f5fac06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- 172.21.15.22:0/2141426359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009e50 tx=0x7f5fa401ca30 comp rx=0 tx=0).stop 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/2141426359 shutdown_connections 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- 172.21.15.22:0/2141426359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac06edf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- 172.21.15.22:0/2141426359 >> v2:172.21.15.183:3300/0 conn(0x7f5fac072740 0x7f5fac072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/2141426359 >> 172.21.15.22:0/2141426359 conn(0x7f5fac067f10 msgr2=0x7f5fac06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/2141426359 shutdown_connections 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/2141426359 wait complete. 2024-04-27T07:53:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 Processor -- start 2024-04-27T07:53:56.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- start start 2024-04-27T07:53:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fac072740 0x7f5fac15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac0aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15d690 con 0x7f5fac1454b0 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15d510 con 0x7f5fac072740 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac15d810 con 0x7f5fac145a00 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47456/0 (socket says 172.21.15.22:47456) 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 -- 172.21.15.22:0/3087912367 learned_addr learned my addr 172.21.15.22:0/3087912367 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fab7fe640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fac072740 0x7f5fac15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fabfff640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac0aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 -- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 msgr2=0x7f5fac0aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac0aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 -- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fac072740 msgr2=0x7f5fac15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fac072740 0x7f5fac15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 -- 172.21.15.22:0/3087912367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac15b920 con 0x7f5fac1454b0 2024-04-27T07:53:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5faaffd640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000ebe0 tx=0x7f5fa000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0017070 con 0x7f5fac1454b0 2024-04-27T07:53:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa000cc30 con 0x7f5fac1454b0 2024-04-27T07:53:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac15bc00 con 0x7f5fac1454b0 2024-04-27T07:53:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0013470 con 0x7f5fac1454b0 2024-04-27T07:53:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.501+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac15a740 con 0x7f5fac1454b0 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac068c30 con 0x7f5fac1454b0 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5fa001a070 con 0x7f5fac1454b0 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5f93fff640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 0x7f5f9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5fa0086ce0 con 0x7f5fac1454b0 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5fab7fe640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 0x7f5f9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.505+0000 7f5fab7fe640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 0x7f5f9806a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001650 tx=0x7f5f9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.509+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5fa00572a0 con 0x7f5fac1454b0 2024-04-27T07:53:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.665+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac0b00d0 con 0x7f5fac1454b0 2024-04-27T07:53:56.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.665+0000 7f5f93fff640 1 -- 172.21.15.22:0/3087912367 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7f5fa0056ba0 con 0x7f5fac1454b0 2024-04-27T07:53:56.671 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:53:56.671 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:51:56.149113+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:51:52.976189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:51:53.432250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 msgr2=0x7f5f9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 0x7f5f9806a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001650 tx=0x7f5f9c009040 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 msgr2=0x7f5fac0a9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000ebe0 tx=0x7f5fa000c520 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 shutdown_connections 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f98068380 0x7f5f9806a840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fac145a00 0x7f5fac0aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fac1454b0 0x7f5fac0a9c30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 --2- 172.21.15.22:0/3087912367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fac072740 0x7f5fac15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 >> 172.21.15.22:0/3087912367 conn(0x7f5fac067f10 msgr2=0x7f5fac13fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 shutdown_connections 2024-04-27T07:53:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.669+0000 7f5fb2212640 1 -- 172.21.15.22:0/3087912367 wait complete. 2024-04-27T07:53:56.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-27T07:53:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 Processor -- start 2024-04-27T07:53:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 -- start start 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80d4790 0x7f63e80d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63e815ebe0 0x7f63e80cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 0x7f63e80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f63e80d4540 con 0x7f63e815ebe0 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f63e80b3d30 con 0x7f63e80b7930 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63ed2c0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80ba7a0 con 0x7f63e80d4790 2024-04-27T07:53:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63e815ebe0 0x7f63e80cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63e815ebe0 0x7f63e80cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47462/0 (socket says 172.21.15.22:47462) 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 -- 172.21.15.22:0/1230094703 learned_addr learned my addr 172.21.15.22:0/1230094703 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e77fe640 1 --2- 172.21.15.22:0/1230094703 >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 0x7f63e80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e6ffd640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80d4790 0x7f63e80d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 -- 172.21.15.22:0/1230094703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80d4790 msgr2=0x7f63e80d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80d4790 0x7f63e80d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 -- 172.21.15.22:0/1230094703 >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 msgr2=0x7f63e80b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 --2- 172.21.15.22:0/1230094703 >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 0x7f63e80b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e67fc640 1 -- 172.21.15.22:0/1230094703 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e80d50b0 con 0x7f63e815ebe0 2024-04-27T07:53:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.797+0000 7f63e77fe640 1 --2- 172.21.15.22:0/1230094703 >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 0x7f63e80b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63e67fc640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e815ebe0 0x7f63e80cf860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f63e80d1250 tx=0x7f63d801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa2f2d0755e421d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63e5ffb640 1 -- 172.21.15.22:0/1230094703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b0070 con 0x7f63e815ebe0 2024-04-27T07:53:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63e5ffb640 1 -- 172.21.15.22:0/1230094703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80ab600 con 0x7f63e815ebe0 2024-04-27T07:53:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63e5ffb640 1 -- 172.21.15.22:0/1230094703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ab9d0 con 0x7f63e815ebe0 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- 172.21.15.22:0/1230094703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e815ebe0 msgr2=0x7f63e80cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e815ebe0 0x7f63e80cf860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f63e80d1250 tx=0x7f63d801c1a0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- 172.21.15.22:0/1230094703 shutdown_connections 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/1230094703 >> v2:172.21.15.183:3300/0 conn(0x7f63e80b7930 0x7f63e80b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e815ebe0 0x7f63e80cf860 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/1230094703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80d4790 0x7f63e80d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- 172.21.15.22:0/1230094703 >> 172.21.15.22:0/1230094703 conn(0x7f63e8067d10 msgr2=0x7f63e8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- 172.21.15.22:0/1230094703 shutdown_connections 2024-04-27T07:53:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- 172.21.15.22:0/1230094703 wait complete. 2024-04-27T07:53:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 Processor -- start 2024-04-27T07:53:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.801+0000 7f63ed2c0640 1 -- start start 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80b7930 0x7f63e8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 0x7f63e814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80ba2a0 con 0x7f63e80d4790 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80ba120 con 0x7f63e814dde0 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80cfda0 con 0x7f63e80b7930 2024-04-27T07:53:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47468/0 (socket says 172.21.15.22:47468) 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 -- 172.21.15.22:0/266079119 learned_addr learned my addr 172.21.15.22:0/266079119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e6ffd640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80b7930 0x7f63e8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e77fe640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 0x7f63e814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 -- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80b7930 msgr2=0x7f63e8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80b7930 0x7f63e8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 -- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 msgr2=0x7f63e814b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 0x7f63e814b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 -- 172.21.15.22:0/266079119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e8150cf0 con 0x7f63e80d4790 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e77fe640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 0x7f63e814b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e67fc640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f63d801a850 tx=0x7f63d8018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b9070 con 0x7f63e80d4790 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80b4400 con 0x7f63e80d4790 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e8150fd0 con 0x7f63e80d4790 2024-04-27T07:53:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b0040 con 0x7f63e80d4790 2024-04-27T07:53:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e8151590 con 0x7f63e80d4790 2024-04-27T07:53:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e8068c30 con 0x7f63e80d4790 2024-04-27T07:53:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f63d80bc070 con 0x7f63e80d4790 2024-04-27T07:53:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 0x7f63d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f63d811fac0 con 0x7f63e80d4790 2024-04-27T07:53:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.805+0000 7f63e6ffd640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 0x7f63d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.809+0000 7f63e6ffd640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 0x7f63d406a870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f63e8068230 tx=0x7f63dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:56.809+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63d8016110 con 0x7f63e80d4790 2024-04-27T07:53:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.001+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f63e80b9bb0 con 0x7f63e80d4790 2024-04-27T07:53:57.004 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-04-27T07:53:57.001+0000 7fadb8a64640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T07:53:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.001+0000 7f63cf7fe640 1 -- 172.21.15.22:0/266079119 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f63d80f0080 con 0x7f63e80d4790 2024-04-27T07:53:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 msgr2=0x7f63d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 0x7f63d406a870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f63e8068230 tx=0x7f63dc008040 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 msgr2=0x7f63e814ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f63d801a850 tx=0x7f63d8018f40 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 shutdown_connections 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f63d40683b0 0x7f63d406a870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e814dde0 0x7f63e814b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63e80d4790 0x7f63e814ad80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 --2- 172.21.15.22:0/266079119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e80b7930 0x7f63e8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 >> 172.21.15.22:0/266079119 conn(0x7f63e8067d10 msgr2=0x7f63e80b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 shutdown_connections 2024-04-27T07:53:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.005+0000 7f63ed2c0640 1 -- 172.21.15.22:0/266079119 wait complete. 2024-04-27T07:53:57.021 INFO:tasks.ceph.mds.c.smithi022.stderr: -7090> 2024-04-27T07:52:00.078+0000 7f5c0f429640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:53:57.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-27T07:53:57.065 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-04-27T07:53:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 Processor -- start 2024-04-27T07:53:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 -- start start 2024-04-27T07:53:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140d1940 0x7f9e140d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140d5680 con 0x7f9e14161fa0 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140cf4d0 con 0x7f9e14163130 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e1a944640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140cfe00 con 0x7f9e140d1940 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e13fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9e14161fa0 0x7f9e140d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44206/0 (socket says 172.21.15.22:44206) 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e13fff640 1 -- 172.21.15.22:0/1109083355 learned_addr learned my addr 172.21.15.22:0/1109083355 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140d1940 0x7f9e140d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612765868 0 0) 0x7f9e140d5680 con 0x7f9e14161fa0 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dfc003530 con 0x7f9e14161fa0 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1640658687 0 0) 0x7f9dfc003530 con 0x7f9e14161fa0 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140d1940 msgr2=0x7f9e140d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140d1940 0x7f9e140d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 >> v1:172.21.15.183:6789/0 conn(0x7f9e14163130 legacy=0x7f9e140b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140cf7b0 con 0x7f9e14161fa0 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f9e04001680 con 0x7f9e14161fa0 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 >> v1:172.21.15.22:6789/0 conn(0x7f9e14161fa0 legacy=0x7f9e140d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9dfc0039f0 0x7f9dfc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9dfc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dfc005220 0x7f9dfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140cfe00 con 0x7f9dfc004810 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc00c120 con 0x7f9dfc005220 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9dfc00c320 con 0x7f9dfc0039f0 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140d1940 0x7f9e140d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9dfc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e18ee1640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dfc005220 0x7f9dfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e13fff640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9dfc0039f0 0x7f9dfc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 -- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9dfc0039f0 msgr2=0x7f9dfc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9dfc0039f0 0x7f9dfc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 -- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dfc005220 msgr2=0x7f9dfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dfc005220 0x7f9dfc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e137fe640 1 -- 172.21.15.22:0/1109083355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc003530 con 0x7f9dfc004810 2024-04-27T07:53:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.133+0000 7f9e13fff640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9dfc0039f0 0x7f9dfc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9dfc004c90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006fd0 tx=0x7f9e0801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877a009b979615f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080b0070 con 0x7f9dfc004810 2024-04-27T07:53:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e080ab600 con 0x7f9dfc004810 2024-04-27T07:53:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e12ffd640 1 -- 172.21.15.22:0/1109083355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800f040 con 0x7f9dfc004810 2024-04-27T07:53:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- 172.21.15.22:0/1109083355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 msgr2=0x7f9dfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1109083355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9dfc004c90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006fd0 tx=0x7f9e0801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e18ee1640 1 -- 172.21.15.22:0/1109083355 reap_dead start 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- 172.21.15.22:0/1109083355 shutdown_connections 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- 172.21.15.22:0/1109083355 >> 172.21.15.22:0/1109083355 conn(0x7f9e1408bfc0 msgr2=0x7f9e140d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- 172.21.15.22:0/1109083355 shutdown_connections 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- 172.21.15.22:0/1109083355 wait complete. 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 Processor -- start 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- start start 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e140b9eb0 0x7f9e1415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 0x7f9e14168a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140b7d80 con 0x7f9dfc004810 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140d5340 con 0x7f9e140b9eb0 2024-04-27T07:53:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e1a944640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140b7f00 con 0x7f9e140cda90 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47498/0 (socket says 172.21.15.22:47498) 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.137+0000 7f9e13fff640 1 -- 172.21.15.22:0/1088100259 learned_addr learned my addr 172.21.15.22:0/1088100259 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e140b9eb0 0x7f9e1415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e18ee1640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 0x7f9e14168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 -- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 msgr2=0x7f9e14168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 0x7f9e14168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 -- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e140b9eb0 msgr2=0x7f9e1415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e140b9eb0 0x7f9e1415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 -- 172.21.15.22:0/1088100259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1415d830 con 0x7f9dfc004810 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e18ee1640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 0x7f9e14168a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e13fff640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400d8e0 tx=0x7f9e0400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04019070 con 0x7f9dfc004810 2024-04-27T07:53:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e040074e0 con 0x7f9dfc004810 2024-04-27T07:53:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e140b9300 con 0x7f9dfc004810 2024-04-27T07:53:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040049f0 con 0x7f9dfc004810 2024-04-27T07:53:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e140ccfb0 con 0x7f9dfc004810 2024-04-27T07:53:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1404ff10 con 0x7f9dfc004810 2024-04-27T07:53:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9e0401c070 con 0x7f9dfc004810 2024-04-27T07:53:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 0x7f9e0006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9e04086490 con 0x7f9dfc004810 2024-04-27T07:53:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 0x7f9e0006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.141+0000 7f9e137fe640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 0x7f9e0006a870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e080ad000 tx=0x7f9e080046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.145+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e04056390 con 0x7f9dfc004810 2024-04-27T07:53:57.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:57.341+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9e140d1010 con 0x7f9dfc004810 2024-04-27T07:53:57.534 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:57.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:58.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.005+0000 7f9e10ff9640 1 -- 172.21.15.22:0/1088100259 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f9e0405a690 con 0x7f9dfc004810 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 msgr2=0x7f9e0006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 0x7f9e0006a870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e080ad000 tx=0x7f9e080046d0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 msgr2=0x7f9e140b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400d8e0 tx=0x7f9e0400dda0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 shutdown_connections 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f9e000683b0 0x7f9e0006a870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9e140cda90 0x7f9e14168a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e140b9eb0 0x7f9e1415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 --2- 172.21.15.22:0/1088100259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc004810 0x7f9e140b8c40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 >> 172.21.15.22:0/1088100259 conn(0x7f9e1408bfc0 msgr2=0x7f9e1406e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:58.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 shutdown_connections 2024-04-27T07:53:58.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.009+0000 7f9e1a944640 1 -- 172.21.15.22:0/1088100259 wait complete. 2024-04-27T07:53:58.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-27T07:53:58.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.165+0000 7f309fab6640 1 Processor -- start 2024-04-27T07:53:58.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.165+0000 7f309fab6640 1 -- start start 2024-04-27T07:53:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309fab6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30980c55e0 0x7f30980b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309fab6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30980c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309fab6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f309809aad0 con 0x7f309809b570 2024-04-27T07:53:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309fab6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f309809b0e0 con 0x7f30980c55e0 2024-04-27T07:53:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309fab6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f30980b3b70 con 0x7f30980d0510 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d852640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30980c55e0 0x7f30980b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309e053640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f309809b570 0x7f30980ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44222/0 (socket says 172.21.15.22:44222) 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309e053640 1 -- 172.21.15.22:0/1830417146 learned_addr learned my addr 172.21.15.22:0/1830417146 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d051640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30980c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d852640 1 -- 172.21.15.22:0/1830417146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 msgr2=0x7f30980c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d852640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30980c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d852640 1 -- 172.21.15.22:0/1830417146 >> v1:172.21.15.22:6789/0 conn(0x7f309809b570 legacy=0x7f30980ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309c850640 1 -- 172.21.15.22:0/1830417146 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338821750 0 0) 0x7f309809aad0 con 0x7f309809b570 2024-04-27T07:53:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.169+0000 7f309d852640 1 -- 172.21.15.22:0/1830417146 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309809ad70 con 0x7f30980c55e0 2024-04-27T07:53:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30980c55e0 0x7f30980b3440 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f30940097f0 tx=0x7f309401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4d385b262c3f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309c850640 1 -- 172.21.15.22:0/1830417146 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940b0070 con 0x7f30980c55e0 2024-04-27T07:53:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309c850640 1 -- 172.21.15.22:0/1830417146 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30940ab600 con 0x7f30980c55e0 2024-04-27T07:53:58.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309c850640 1 -- 172.21.15.22:0/1830417146 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940ab9d0 con 0x7f30980c55e0 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- 172.21.15.22:0/1830417146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30980c55e0 msgr2=0x7f30980b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30980c55e0 0x7f30980b3440 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f30940097f0 tx=0x7f309401c4a0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- 172.21.15.22:0/1830417146 shutdown_connections 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30980c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- 172.21.15.22:0/1830417146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30980c55e0 0x7f30980b3440 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- 172.21.15.22:0/1830417146 >> 172.21.15.22:0/1830417146 conn(0x7f30980b4100 msgr2=0x7f30980b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:58.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- 172.21.15.22:0/1830417146 shutdown_connections 2024-04-27T07:53:58.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- 172.21.15.22:0/1830417146 wait complete. 2024-04-27T07:53:58.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 Processor -- start 2024-04-27T07:53:58.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- start start 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30980c55e0 0x7f30980af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30981615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f309809b9c0 con 0x7f309809b570 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30980b3e50 con 0x7f30980c55e0 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309fab6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f309809bb40 con 0x7f30980d0510 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47522/0 (socket says 172.21.15.22:47522) 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 -- 172.21.15.22:0/3862442528 learned_addr learned my addr 172.21.15.22:0/3862442528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d051640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30980c55e0 0x7f30980af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309e053640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30981615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 -- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 msgr2=0x7f30981615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30981615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 -- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30980c55e0 msgr2=0x7f30980af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30980c55e0 0x7f30980af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.173+0000 7f309d852640 1 -- 172.21.15.22:0/3862442528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30980afda0 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309d852640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f30940ae040 tx=0x7f309401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940b0050 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30940b1040 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30980b0080 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004240 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30981566f0 con 0x7f309809b570 2024-04-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309804ff10 con 0x7f309809b570 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f30940be060 con 0x7f309809b570 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 0x7f307c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f3094120510 con 0x7f309809b570 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309d051640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 0x7f307c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.177+0000 7f309d051640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 0x7f307c06a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30980c5a70 tx=0x7f3088008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.181+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3094016090 con 0x7f309809b570 2024-04-27T07:53:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:58.345+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f309806e9d0 con 0x7f309809b570 2024-04-27T07:53:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.017+0000 7f308e7fc640 1 -- 172.21.15.22:0/3862442528 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f30940bf3b0 con 0x7f309809b570 2024-04-27T07:53:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-04-27T07:53:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.017+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 msgr2=0x7f307c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.017+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 0x7f307c06a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30980c5a70 tx=0x7f3088008040 comp rx=0 tx=0).stop 2024-04-27T07:53:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.017+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 msgr2=0x7f30981561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:59.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.017+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f30940ae040 tx=0x7f309401ace0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 shutdown_connections 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f307c068380 0x7f307c06a840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30980d0510 0x7f30981615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30980c55e0 0x7f30980af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 --2- 172.21.15.22:0/3862442528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f309809b570 0x7f30981561b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 >> 172.21.15.22:0/3862442528 conn(0x7f30980b4100 msgr2=0x7f3098071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 shutdown_connections 2024-04-27T07:53:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.021+0000 7f309fab6640 1 -- 172.21.15.22:0/3862442528 wait complete. 2024-04-27T07:53:59.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-27T07:53:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 Processor -- start 2024-04-27T07:53:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 -- start start 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd74015ecd0 0x7fd740072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd7400cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7400b5390 con 0x7fd7400b4ee0 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd7400b5670 con 0x7fd74015ecd0 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.177+0000 7fd748539640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd7400ce340 con 0x7fd7400cee20 2024-04-27T07:53:59.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd74015ecd0 0x7fd740072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd74015ecd0 0x7fd740072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47532/0 (socket says 172.21.15.22:47532) 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 -- 172.21.15.22:0/805416813 learned_addr learned my addr 172.21.15.22:0/805416813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd746ad6640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd7400cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 -- 172.21.15.22:0/805416813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 msgr2=0x7fd7400cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd7400cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 -- 172.21.15.22:0/805416813 >> v1:172.21.15.183:6789/0 conn(0x7fd7400b4ee0 legacy=0x7fd7400b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:53:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 -- 172.21.15.22:0/805416813 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7400ce620 con 0x7fd74015ecd0 2024-04-27T07:53:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7462d5640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd74015ecd0 0x7fd740072980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd73400bb10 tx=0x7fd73401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93f05964c1772ce server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7452d3640 1 -- 172.21.15.22:0/805416813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340b4070 con 0x7fd74015ecd0 2024-04-27T07:53:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7452d3640 1 -- 172.21.15.22:0/805416813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7340a7b70 con 0x7fd74015ecd0 2024-04-27T07:53:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd7452d3640 1 -- 172.21.15.22:0/805416813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340af440 con 0x7fd74015ecd0 2024-04-27T07:53:59.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 -- 172.21.15.22:0/805416813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd74015ecd0 msgr2=0x7fd740072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd74015ecd0 0x7fd740072980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd73400bb10 tx=0x7fd73401ebc0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 -- 172.21.15.22:0/805416813 shutdown_connections 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd7400cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 --2- 172.21.15.22:0/805416813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd74015ecd0 0x7fd740072980 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 -- 172.21.15.22:0/805416813 >> 172.21.15.22:0/805416813 conn(0x7fd7400b21d0 msgr2=0x7fd7400b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 -- 172.21.15.22:0/805416813 shutdown_connections 2024-04-27T07:53:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.181+0000 7fd748539640 1 -- 172.21.15.22:0/805416813 wait complete. 2024-04-27T07:53:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 Processor -- start 2024-04-27T07:53:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- start start 2024-04-27T07:53:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd740156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74015ecd0 0x7fd7400d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006e960 con 0x7fd7400b4ee0 2024-04-27T07:53:59.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006e7e0 con 0x7fd74015ecd0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd74006eae0 con 0x7fd7400cee20 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47548/0 (socket says 172.21.15.22:47548) 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 -- 172.21.15.22:0/3624338068 learned_addr learned my addr 172.21.15.22:0/3624338068 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd745ad4640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd740156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd746ad6640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74015ecd0 0x7fd7400d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 -- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 msgr2=0x7fd740156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd740156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 -- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74015ecd0 msgr2=0x7fd7400d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74015ecd0 0x7fd7400d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 -- 172.21.15.22:0/3624338068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd740140f10 con 0x7fd7400b4ee0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd7462d5640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd73400b2c0 tx=0x7fd73401d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:59.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340bf070 con 0x7fd7400b4ee0 2024-04-27T07:53:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7340ba440 con 0x7fd7400b4ee0 2024-04-27T07:53:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd740158c90 con 0x7fd7400b4ee0 2024-04-27T07:53:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340b4070 con 0x7fd7400b4ee0 2024-04-27T07:53:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.185+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd740159250 con 0x7fd7400b4ee0 2024-04-27T07:53:59.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74004ff10 con 0x7fd7400b4ee0 2024-04-27T07:53:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd7340c2070 con 0x7fd7400b4ee0 2024-04-27T07:53:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd732ffd640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 0x7fd72406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:53:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7fd734120ee0 con 0x7fd7400b4ee0 2024-04-27T07:53:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd745ad4640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 0x7fd72406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:53:59.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.189+0000 7fd745ad4640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 0x7fd72406a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7401fafa0 tx=0x7fd73c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:53:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.193+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7340187e0 con 0x7fd7400b4ee0 2024-04-27T07:53:59.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:53:59.361+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd7400cfc60 con 0x7fd7400b4ee0 2024-04-27T07:54:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.017+0000 7fd732ffd640 1 -- 172.21.15.22:0/3624338068 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7fd7340f1690 con 0x7fd7400b4ee0 2024-04-27T07:54:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-04-27T07:54:00.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 msgr2=0x7fd72406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 0x7fd72406a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7401fafa0 tx=0x7fd73c009040 comp rx=0 tx=0).stop 2024-04-27T07:54:00.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 msgr2=0x7fd7401409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd73400b2c0 tx=0x7fd73401d400 comp rx=0 tx=0).stop 2024-04-27T07:54:00.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 shutdown_connections 2024-04-27T07:54:00.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fd724068380 0x7fd72406a840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74015ecd0 0x7fd7400d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7400cee20 0x7fd740156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 --2- 172.21.15.22:0/3624338068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7400b4ee0 0x7fd7401409d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 >> 172.21.15.22:0/3624338068 conn(0x7fd7400b21d0 msgr2=0x7fd74015a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:00.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 shutdown_connections 2024-04-27T07:54:00.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.025+0000 7fd748539640 1 -- 172.21.15.22:0/3624338068 wait complete. 2024-04-27T07:54:00.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-27T07:54:00.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 Processor -- start 2024-04-27T07:54:00.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 -- start start 2024-04-27T07:54:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f24c8162f60 0x7f24c809b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c806cce0 0x7f24c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24c80c5610 0x7f24c80c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f24c80c5fe0 con 0x7f24c80c5610 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f24c806d730 con 0x7f24c8162f60 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24d044d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f24c809abb0 con 0x7f24c806cce0 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24c80c5610 0x7f24c80c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24c80c5610 0x7f24c80c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57614/0 (socket says 172.21.15.22:57614) 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 -- 172.21.15.22:0/1998350512 learned_addr learned my addr 172.21.15.22:0/1998350512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24cd9e8640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c806cce0 0x7f24c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce1e9640 1 --2- 172.21.15.22:0/1998350512 >> v2:172.21.15.183:3300/0 conn(0x7f24c8162f60 0x7f24c809b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 -- 172.21.15.22:0/1998350512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c806cce0 msgr2=0x7f24c806d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c806cce0 0x7f24c806d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 -- 172.21.15.22:0/1998350512 >> v2:172.21.15.183:3300/0 conn(0x7f24c8162f60 msgr2=0x7f24c809b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 --2- 172.21.15.22:0/1998350512 >> v2:172.21.15.183:3300/0 conn(0x7f24c8162f60 0x7f24c809b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.177+0000 7f24ce9ea640 1 -- 172.21.15.22:0/1998350512 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c809ae90 con 0x7f24c80c5610 2024-04-27T07:54:00.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24ce9ea640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c80c5610 0x7f24c80c5aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f24c8073800 tx=0x7f24c001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6068dbd386ce70c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24cd1e7640 1 -- 172.21.15.22:0/1998350512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00a57b0 con 0x7f24c80c5610 2024-04-27T07:54:00.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24cd1e7640 1 -- 172.21.15.22:0/1998350512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c00a5950 con 0x7f24c80c5610 2024-04-27T07:54:00.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24cd1e7640 1 -- 172.21.15.22:0/1998350512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00a5d20 con 0x7f24c80c5610 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 -- 172.21.15.22:0/1998350512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c80c5610 msgr2=0x7f24c80c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c80c5610 0x7f24c80c5aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f24c8073800 tx=0x7f24c001c930 comp rx=0 tx=0).stop 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 -- 172.21.15.22:0/1998350512 shutdown_connections 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c80c5610 0x7f24c80c5aa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 --2- 172.21.15.22:0/1998350512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c806cce0 0x7f24c806d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 --2- 172.21.15.22:0/1998350512 >> v2:172.21.15.183:3300/0 conn(0x7f24c8162f60 0x7f24c809b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.181+0000 7f24d044d640 1 -- 172.21.15.22:0/1998350512 >> 172.21.15.22:0/1998350512 conn(0x7f24c80b4130 msgr2=0x7f24c80b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:00.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- 172.21.15.22:0/1998350512 shutdown_connections 2024-04-27T07:54:00.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- 172.21.15.22:0/1998350512 wait complete. 2024-04-27T07:54:00.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 Processor -- start 2024-04-27T07:54:00.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- start start 2024-04-27T07:54:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c80c5610 0x7f24c80c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24c8162f60 0x7f24c80b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d0ba0 con 0x7f24c806cce0 2024-04-27T07:54:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d0a20 con 0x7f24c8162f60 2024-04-27T07:54:00.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.185+0000 7f24d044d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80d0d20 con 0x7f24c80c5610 2024-04-27T07:54:00.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce9ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24c8162f60 0x7f24c80b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57626/0 (socket says 172.21.15.22:57626) 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 -- 172.21.15.22:0/4079993672 learned_addr learned my addr 172.21.15.22:0/4079993672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24cd9e8640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c80c5610 0x7f24c80c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 -- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c80c5610 msgr2=0x7f24c80c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c80c5610 0x7f24c80c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 -- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24c8162f60 msgr2=0x7f24c80b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24c8162f60 0x7f24c80b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 -- 172.21.15.22:0/4079993672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80c93f0 con 0x7f24c806cce0 2024-04-27T07:54:00.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24ce1e9640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e870 tx=0x7f24c400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017070 con 0x7f24c806cce0 2024-04-27T07:54:00.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c400c940 con 0x7f24c806cce0 2024-04-27T07:54:00.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80c9110 con 0x7f24c806cce0 2024-04-27T07:54:00.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80c8d30 con 0x7f24c806cce0 2024-04-27T07:54:00.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400ccf0 con 0x7f24c806cce0 2024-04-27T07:54:00.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b00052a0 con 0x7f24c806cce0 2024-04-27T07:54:00.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f24c401a070 con 0x7f24c806cce0 2024-04-27T07:54:00.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.189+0000 7f24beffd640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 0x7f24ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.193+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f24c40863f0 con 0x7f24c806cce0 2024-04-27T07:54:00.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.193+0000 7f24cd9e8640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 0x7f24ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.193+0000 7f24cd9e8640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 0x7f24ac06a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24c816dc50 tx=0x7f24b8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.197+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24c40566f0 con 0x7f24c806cce0 2024-04-27T07:54:00.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.365+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f24b0005860 con 0x7f24c806cce0 2024-04-27T07:54:00.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24beffd640 1 -- 172.21.15.22:0/4079993672 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f24c405a9f0 con 0x7f24c806cce0 2024-04-27T07:54:00.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 msgr2=0x7f24ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 0x7f24ac06a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24c816dc50 tx=0x7f24b8009190 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 msgr2=0x7f24c8157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e870 tx=0x7f24c400ec70 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 shutdown_connections 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f24ac068380 0x7f24ac06a840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24c8162f60 0x7f24c80b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24c80c5610 0x7f24c80c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 --2- 172.21.15.22:0/4079993672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24c806cce0 0x7f24c8157340 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 >> 172.21.15.22:0/4079993672 conn(0x7f24c80b4130 msgr2=0x7f24c80b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 shutdown_connections 2024-04-27T07:54:00.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.369+0000 7f24d044d640 1 -- 172.21.15.22:0/4079993672 wait complete. 2024-04-27T07:54:00.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-27T07:54:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.505+0000 7faf827ab640 1 Processor -- start 2024-04-27T07:54:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.505+0000 7faf827ab640 1 -- start start 2024-04-27T07:54:00.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf827ab640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf7c144dd0 0x7faf7c06f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf827ab640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c0720d0 con 0x7faf7c0cfe10 2024-04-27T07:54:00.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf827ab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c072370 con 0x7faf7c06f6a0 2024-04-27T07:54:00.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf827ab640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c06fb00 con 0x7faf7c144dd0 2024-04-27T07:54:00.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf7c144dd0 0x7faf7c06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf7c144dd0 0x7faf7c06f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57640/0 (socket says 172.21.15.22:57640) 2024-04-27T07:54:00.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 -- 172.21.15.22:0/3499725283 learned_addr learned my addr 172.21.15.22:0/3499725283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:00.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 -- 172.21.15.22:0/3499725283 >> v1:172.21.15.183:6789/0 conn(0x7faf7c06f6a0 legacy=0x7faf7c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:00.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 -- 172.21.15.22:0/3499725283 >> v1:172.21.15.22:6789/0 conn(0x7faf7c0cfe10 legacy=0x7faf7c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 -- 172.21.15.22:0/3499725283 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c072650 con 0x7faf7c144dd0 2024-04-27T07:54:00.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7affd640 1 -- 172.21.15.22:0/3499725283 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917714451 0 0) 0x7faf7c0720d0 con 0x7faf7c0cfe10 2024-04-27T07:54:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.509+0000 7faf7bfff640 1 --2- 172.21.15.22:0/3499725283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c06f160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddc9a2da4ad9fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf7affd640 1 -- 172.21.15.22:0/3499725283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0a5700 con 0x7faf7c144dd0 2024-04-27T07:54:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf7affd640 1 -- 172.21.15.22:0/3499725283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c0a58a0 con 0x7faf7c144dd0 2024-04-27T07:54:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf7affd640 1 -- 172.21.15.22:0/3499725283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0a5c70 con 0x7faf7c144dd0 2024-04-27T07:54:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/3499725283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 msgr2=0x7faf7c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 --2- 172.21.15.22:0/3499725283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c06f160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c01ca30 comp rx=0 tx=0).stop 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/3499725283 shutdown_connections 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 --2- 172.21.15.22:0/3499725283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c06f160 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/3499725283 >> 172.21.15.22:0/3499725283 conn(0x7faf7c067f10 msgr2=0x7faf7c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/3499725283 shutdown_connections 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/3499725283 wait complete. 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 Processor -- start 2024-04-27T07:54:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- start start 2024-04-27T07:54:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf7c06f6a0 0x7faf7c14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf7c0cfe10 0x7faf7c0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c0b4e70 con 0x7faf7c144dd0 2024-04-27T07:54:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c0b4cf0 con 0x7faf7c0cfe10 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7faf7c0b4ff0 con 0x7faf7c06f6a0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57650/0 (socket says 172.21.15.22:57650) 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 -- 172.21.15.22:0/1135650963 learned_addr learned my addr 172.21.15.22:0/1135650963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf7b7fe640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf7c0cfe10 0x7faf7c0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf7bfff640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf7c06f6a0 0x7faf7c14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 -- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf7c06f6a0 msgr2=0x7faf7c14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf7c06f6a0 0x7faf7c14e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 -- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf7c0cfe10 msgr2=0x7faf7c0c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf7c0cfe10 0x7faf7c0c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 -- 172.21.15.22:0/1135650963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0c73a0 con 0x7faf7c144dd0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf80d48640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c4b0 tx=0x7faf7400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740052d0 con 0x7faf7c144dd0 2024-04-27T07:54:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74005490 con 0x7faf7c144dd0 2024-04-27T07:54:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c0c6300 con 0x7faf7c144dd0 2024-04-27T07:54:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74012850 con 0x7faf7c144dd0 2024-04-27T07:54:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.513+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c0c68c0 con 0x7faf7c144dd0 2024-04-27T07:54:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c068c30 con 0x7faf7c144dd0 2024-04-27T07:54:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faf740129f0 con 0x7faf7c144dd0 2024-04-27T07:54:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf78ff9640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 0x7faf6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7faf74086260 con 0x7faf7c144dd0 2024-04-27T07:54:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf7bfff640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 0x7faf6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.517+0000 7faf7bfff640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 0x7faf6806a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0ae000 tx=0x7faf6c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.521+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf7c0c73a0 con 0x7faf7c144dd0 2024-04-27T07:54:00.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:00.745+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7faf7c0cc820 con 0x7faf7c144dd0 2024-04-27T07:54:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.029+0000 7faf78ff9640 1 -- 172.21.15.22:0/1135650963 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v148) ==== 120+0+0 (secure 0 0 0) 0x7faf74056b20 con 0x7faf7c144dd0 2024-04-27T07:54:01.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 msgr2=0x7faf6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:01.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 0x7faf6806a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0ae000 tx=0x7faf6c0a7070 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 msgr2=0x7faf7c0cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c4b0 tx=0x7faf7400c970 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 shutdown_connections 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7faf68068380 0x7faf6806a840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf7c144dd0 0x7faf7c0cb9f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf7c0cfe10 0x7faf7c0c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 --2- 172.21.15.22:0/1135650963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf7c06f6a0 0x7faf7c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 >> 172.21.15.22:0/1135650963 conn(0x7faf7c067f10 msgr2=0x7faf7c140390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 shutdown_connections 2024-04-27T07:54:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.033+0000 7faf827ab640 1 -- 172.21.15.22:0/1135650963 wait complete. 2024-04-27T07:54:01.060 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-04-27T07:54:01.061 INFO:tasks.cephfs_test_runner: 2024-04-27T07:54:01.061 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-27T07:54:01.061 INFO:tasks.cephfs_test_runner:Ran 1 test in 156.279s 2024-04-27T07:54:01.061 INFO:tasks.cephfs_test_runner: 2024-04-27T07:54:01.061 INFO:tasks.cephfs_test_runner:OK 2024-04-27T07:54:01.061 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-27T07:54:01.071 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-27T07:54:01.083 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-27T07:54:01.083 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:54:01.083 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:54:01.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21058 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-27T07:54:01.090 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-27T07:54:01.090 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:54:01.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21058 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-27T07:54:01.136 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:54:01.136 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:54:01.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16118 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-27T07:54:01.143 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-27T07:54:01.143 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:54:01.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16118 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-27T07:54:01.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.190 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2024-04-27T07:54:01.238 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-27T07:54:01.238 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-27T07:54:01.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-27T07:54:01.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-27T07:54:01.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2024-04-27T07:54:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.538 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-27T07:54:01.545 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-27T07:54:01.545 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T07:54:01.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-27T07:54:01.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:54:01.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:54:01.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-04-27T07:54:01.820 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-27T07:54:01.830 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T07:54:01.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:54:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.953+0000 7f3031109640 1 Processor -- start 2024-04-27T07:54:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.953+0000 7f3031109640 1 -- start start 2024-04-27T07:54:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3031109640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f302c15ed30 0x7f302c06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3031109640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f302c09aa30 0x7f302c09ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3031109640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302c0d1520 con 0x7f302c162680 2024-04-27T07:54:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3031109640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f302c09b3d0 con 0x7f302c15ed30 2024-04-27T07:54:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3031109640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f302c06d6f0 con 0x7f302c09aa30 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f302c15ed30 0x7f302c06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f302c15ed30 0x7f302c06cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57654/0 (socket says 172.21.15.22:57654) 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 -- 172.21.15.22:0/1297615768 learned_addr learned my addr 172.21.15.22:0/1297615768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302a59c640 1 --2- 172.21.15.22:0/1297615768 >> v2:172.21.15.183:3300/0 conn(0x7f302c09aa30 0x7f302c09ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 -- 172.21.15.22:0/1297615768 >> v2:172.21.15.183:3300/0 conn(0x7f302c09aa30 msgr2=0x7f302c09ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 --2- 172.21.15.22:0/1297615768 >> v2:172.21.15.183:3300/0 conn(0x7f302c09aa30 0x7f302c09ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 -- 172.21.15.22:0/1297615768 >> v1:172.21.15.183:6789/0 conn(0x7f302c162680 legacy=0x7f302c162a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:01.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 -- 172.21.15.22:0/1297615768 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c06d940 con 0x7f302c15ed30 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f302ad9d640 1 --2- 172.21.15.22:0/1297615768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c15ed30 0x7f302c06cef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f302c072ba0 tx=0x7f301c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d00cae7ca11cba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3029d9b640 1 -- 172.21.15.22:0/1297615768 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0b0070 con 0x7f302c15ed30 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3029d9b640 1 -- 172.21.15.22:0/1297615768 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c0ab600 con 0x7f302c15ed30 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.957+0000 7f3029d9b640 1 -- 172.21.15.22:0/1297615768 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0ab9d0 con 0x7f302c15ed30 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/1297615768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c15ed30 msgr2=0x7f302c06cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- 172.21.15.22:0/1297615768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c15ed30 0x7f302c06cef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f302c072ba0 tx=0x7f301c01c4e0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/1297615768 shutdown_connections 2024-04-27T07:54:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- 172.21.15.22:0/1297615768 >> v2:172.21.15.183:3300/0 conn(0x7f302c09aa30 0x7f302c09ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- 172.21.15.22:0/1297615768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c15ed30 0x7f302c06cef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/1297615768 >> 172.21.15.22:0/1297615768 conn(0x7f302c0b40b0 msgr2=0x7f302c0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/1297615768 shutdown_connections 2024-04-27T07:54:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/1297615768 wait complete. 2024-04-27T07:54:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 Processor -- start 2024-04-27T07:54:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- start start 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 0x7f302c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f302c162680 0x7f302c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0cdda0 con 0x7f302c09aa30 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0cdc20 con 0x7f302c162680 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f302c09b920 con 0x7f302c15ed30 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57656/0 (socket says 172.21.15.22:57656) 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 -- 172.21.15.22:0/3574588725 learned_addr learned my addr 172.21.15.22:0/3574588725 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302b59e640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f302c162680 0x7f302c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302a59c640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 0x7f302c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 -- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 msgr2=0x7f302c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 0x7f302c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 -- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f302c162680 msgr2=0x7f302c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f302c162680 0x7f302c0b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 -- 172.21.15.22:0/3574588725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c155e60 con 0x7f302c09aa30 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302a59c640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 0x7f302c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f302ad9d640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f301c0ae040 tx=0x7f301c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0ba020 con 0x7f302c09aa30 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c0b6420 con 0x7f302c09aa30 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c159190 con 0x7f302c09aa30 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0b0040 con 0x7f302c09aa30 2024-04-27T07:54:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.961+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c158e80 con 0x7f302c09aa30 2024-04-27T07:54:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c069340 con 0x7f302c09aa30 2024-04-27T07:54:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f301c0bd020 con 0x7f302c09aa30 2024-04-27T07:54:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f30137fe640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f301c120820 con 0x7f302c09aa30 2024-04-27T07:54:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f302a59c640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.965+0000 7f302a59c640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 0x7f301806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f302c06e700 tx=0x7f3020008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:01.969+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f301c0c4030 con 0x7f302c09aa30 2024-04-27T07:54:02.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.129+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f302c0b4d10 con 0x7f3018068380 2024-04-27T07:54:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f30137fe640 1 -- 172.21.15.22:0/3574588725 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13446 (secure 0 0 0) 0x7f302c0b4d10 con 0x7f3018068380 2024-04-27T07:54:02.135 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 msgr2=0x7f301806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 0x7f301806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f302c06e700 tx=0x7f3020008040 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 msgr2=0x7f302c155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f301c0ae040 tx=0x7f301c01ad20 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 shutdown_connections 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f302c162680 0x7f302c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f302c15ed30 0x7f302c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 --2- 172.21.15.22:0/3574588725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f302c09aa30 0x7f302c155880 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 >> 172.21.15.22:0/3574588725 conn(0x7f302c0b40b0 msgr2=0x7f302c162a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.133+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 shutdown_connections 2024-04-27T07:54:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.137+0000 7f3031109640 1 -- 172.21.15.22:0/3574588725 wait complete. 2024-04-27T07:54:02.156 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-27T07:54:00.216664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241184,"kb_used_data":4220,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733536,"statfs":{"total":773094113280,"available":772847140864,"internally_reserved":0,"allocated":4321280,"data_stored":2401181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002031"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-27T07:53:57.015565+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:53:57.015565+0000","last_peered":"2024-04-27T07:53:57.015565+0000","last_clean":"2024-04-27T07:53:57.015565+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:53:57.015565+0000","last_undegraded":"2024-04-27T07:53:57.015565+0000","last_fullsized":"2024-04-27T07:53:57.015565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":740295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":149941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":162976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":385024,"data_stored":149907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":401408,"data_stored":156895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":974848,"data_stored":740205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:54:02.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:54:02.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 Processor -- start 2024-04-27T07:54:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 -- start start 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0070166200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700b3cc0 con 0x7f00700aefb0 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007006f9a0 con 0x7f00700b5490 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0078469640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f007006fc80 con 0x7f00700afaa0 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f00700aefb0 0x7f007006ef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50616/0 (socket says 172.21.15.22:50616) 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 -- 172.21.15.22:0/1305149124 learned_addr learned my addr 172.21.15.22:0/1305149124 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076a06640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0070166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027244500 0 0) 0x7f00700b3cc0 con 0x7f00700aefb0 2024-04-27T07:54:02.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0058003530 con 0x7f00700aefb0 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3300247097 0 0) 0x7f0058003530 con 0x7f00700aefb0 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 msgr2=0x7f0070166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0070166200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 >> v1:172.21.15.183:6789/0 conn(0x7f00700b5490 legacy=0x7f00700af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00700b3f60 con 0x7f00700aefb0 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0064001680 con 0x7f00700aefb0 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 >> v1:172.21.15.22:6789/0 conn(0x7f00700aefb0 legacy=0x7f007006ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00580039f0 0x7f0058003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0058004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0058004d50 0x7f00580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f007006fc80 con 0x7f00580039f0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f005800bb00 con 0x7f0058004d50 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f005800bd00 con 0x7f00700afaa0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00580039f0 0x7f0058003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075a04640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0058004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 -- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 msgr2=0x7f0058004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076a06640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0058004d50 0x7f00580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0058004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 -- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0058004d50 msgr2=0x7f00580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0058004d50 0x7f00580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0076205640 1 -- 172.21.15.22:0/1305149124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0058003530 con 0x7f00580039f0 2024-04-27T07:54:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.277+0000 7f0075a04640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00700afaa0 0x7f0058004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:02.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0076205640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00580039f0 0x7f0058003e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f006400c1c0 tx=0x7f006401d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4bc52213c81290 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064007d00 con 0x7f00580039f0 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00640a9020 con 0x7f00580039f0 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0075203640 1 -- 172.21.15.22:0/1305149124 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064005ae0 con 0x7f00580039f0 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- 172.21.15.22:0/1305149124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00580039f0 msgr2=0x7f0058003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 --2- 172.21.15.22:0/1305149124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00580039f0 0x7f0058003e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f006400c1c0 tx=0x7f006401d5a0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0076a06640 1 -- 172.21.15.22:0/1305149124 reap_dead start 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- 172.21.15.22:0/1305149124 shutdown_connections 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- 172.21.15.22:0/1305149124 >> 172.21.15.22:0/1305149124 conn(0x7f00700d0c30 msgr2=0x7f00700d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- 172.21.15.22:0/1305149124 shutdown_connections 2024-04-27T07:54:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- 172.21.15.22:0/1305149124 wait complete. 2024-04-27T07:54:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 Processor -- start 2024-04-27T07:54:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.281+0000 7f0078469640 1 -- start start 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 0x7f007014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007006bbd0 0x7f007006c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0070166a10 con 0x7f0070157ec0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f007006f610 con 0x7f00580039f0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0070166b90 con 0x7f007006bbd0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57676/0 (socket says 172.21.15.22:57676) 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 -- 172.21.15.22:0/184188413 learned_addr learned my addr 172.21.15.22:0/184188413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0076a06640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007006bbd0 0x7f007006c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0076205640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 0x7f007014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 -- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007006bbd0 msgr2=0x7f007006c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007006bbd0 0x7f007006c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 -- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 msgr2=0x7f007014ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 0x7f007014ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 -- 172.21.15.22:0/184188413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070159010 con 0x7f0070157ec0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0076205640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 0x7f007014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0075a04640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f006000dae0 tx=0x7f0060002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060003db0 con 0x7f0070157ec0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0060017070 con 0x7f0070157ec0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007014ef70 con 0x7f0070157ec0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000b930 con 0x7f0070157ec0 2024-04-27T07:54:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701536d0 con 0x7f0070157ec0 2024-04-27T07:54:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070069340 con 0x7f0070157ec0 2024-04-27T07:54:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f006000bad0 con 0x7f0070157ec0 2024-04-27T07:54:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 0x7f005c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.285+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0060086860 con 0x7f0070157ec0 2024-04-27T07:54:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.289+0000 7f0076205640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 0x7f005c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.289+0000 7f0076205640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 0x7f005c06a870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00640aa000 tx=0x7f00700b4130 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.293+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0060057120 con 0x7f0070157ec0 2024-04-27T07:54:02.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.453+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0070075530 con 0x7f005c0683b0 2024-04-27T07:54:02.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.453+0000 7f006effd640 1 -- 172.21.15.22:0/184188413 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13480 (secure 0 0 0) 0x7f0070075530 con 0x7f005c0683b0 2024-04-27T07:54:02.458 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:02.458 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 msgr2=0x7f005c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 0x7f005c06a870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00640aa000 tx=0x7f00700b4130 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 msgr2=0x7f0070152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f006000dae0 tx=0x7f0060002090 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 shutdown_connections 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f005c0683b0 0x7f005c06a870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007006bbd0 0x7f007006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0070157ec0 0x7f0070152b70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 --2- 172.21.15.22:0/184188413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580039f0 0x7f007014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 >> 172.21.15.22:0/184188413 conn(0x7f00700d0c30 msgr2=0x7f00700af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 shutdown_connections 2024-04-27T07:54:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.457+0000 7f0078469640 1 -- 172.21.15.22:0/184188413 wait complete. 2024-04-27T07:54:02.478 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-04-27T07:54:02.217038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241432,"kb_used_data":4448,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733288,"statfs":{"total":773094113280,"available":772846886912,"internally_reserved":0,"allocated":4554752,"data_stored":2503065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001775"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-27T07:53:57.015565+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:53:57.015565+0000","last_peered":"2024-04-27T07:53:57.015565+0000","last_clean":"2024-04-27T07:53:57.015565+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:53:57.015565+0000","last_undegraded":"2024-04-27T07:53:57.015565+0000","last_fullsized":"2024-04-27T07:53:57.015565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":740295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:54:02.478 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T07:54:02.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:54:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 Processor -- start 2024-04-27T07:54:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 -- start start 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f900806f870 0x7f90080cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90080cfc40 0x7f90080b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90080720d0 con 0x7f9008144dd0 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9008072370 con 0x7f90080cfc40 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f900d30b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9008072650 con 0x7f900806f870 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f90077fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90080cfc40 0x7f90080b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f90077fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90080cfc40 0x7f90080b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57682/0 (socket says 172.21.15.22:57682) 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.589+0000 7f90077fe640 1 -- 172.21.15.22:0/282008893 learned_addr learned my addr 172.21.15.22:0/282008893 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90067fc640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f900806f870 0x7f90080cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90077fe640 1 -- 172.21.15.22:0/282008893 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f900806f870 msgr2=0x7f90080cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90077fe640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f900806f870 0x7f90080cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90077fe640 1 -- 172.21.15.22:0/282008893 >> v1:172.21.15.183:6789/0 conn(0x7f9008144dd0 legacy=0x7f900806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90077fe640 1 -- 172.21.15.22:0/282008893 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9008072930 con 0x7f90080cfc40 2024-04-27T07:54:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f90077fe640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90080cfc40 0x7f90080b3dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9000009870 tx=0x7f900001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3518ae66fde25c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f9005ffb640 1 -- 172.21.15.22:0/282008893 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000ab430 con 0x7f90080cfc40 2024-04-27T07:54:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f9005ffb640 1 -- 172.21.15.22:0/282008893 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000b0070 con 0x7f90080cfc40 2024-04-27T07:54:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f9005ffb640 1 -- 172.21.15.22:0/282008893 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000ab7f0 con 0x7f90080cfc40 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 -- 172.21.15.22:0/282008893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90080cfc40 msgr2=0x7f90080b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90080cfc40 0x7f90080b3dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9000009870 tx=0x7f900001c560 comp rx=0 tx=0).stop 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 -- 172.21.15.22:0/282008893 shutdown_connections 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90080cfc40 0x7f90080b3dd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 --2- 172.21.15.22:0/282008893 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f900806f870 0x7f90080cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 -- 172.21.15.22:0/282008893 >> 172.21.15.22:0/282008893 conn(0x7f9008067f10 msgr2=0x7f900806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 -- 172.21.15.22:0/282008893 shutdown_connections 2024-04-27T07:54:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.593+0000 7f900d30b640 1 -- 172.21.15.22:0/282008893 wait complete. 2024-04-27T07:54:02.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 Processor -- start 2024-04-27T07:54:02.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- start start 2024-04-27T07:54:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900806f870 0x7f90080af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90080cfc40 0x7f900814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90080b4ca0 con 0x7f9008144dd0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90080b4b20 con 0x7f900806f870 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f90080b4e20 con 0x7f90080cfc40 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57692/0 (socket says 172.21.15.22:57692) 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 -- 172.21.15.22:0/1935046897 learned_addr learned my addr 172.21.15.22:0/1935046897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90067fc640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90080cfc40 0x7f900814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f9006ffd640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900806f870 0x7f90080af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 -- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90080cfc40 msgr2=0x7f900814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90080cfc40 0x7f900814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 -- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900806f870 msgr2=0x7f90080af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900806f870 0x7f90080af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 -- 172.21.15.22:0/1935046897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90080ae730 con 0x7f9008144dd0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f90077fe640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f900001a190 tx=0x7f9000019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000bd020 con 0x7f9008144dd0 2024-04-27T07:54:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000b43f0 con 0x7f9008144dd0 2024-04-27T07:54:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90080aea10 con 0x7f9008144dd0 2024-04-27T07:54:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000b47c0 con 0x7f9008144dd0 2024-04-27T07:54:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90080aefd0 con 0x7f9008144dd0 2024-04-27T07:54:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.597+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9008068c30 con 0x7f9008144dd0 2024-04-27T07:54:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.601+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f90000c0070 con 0x7f9008144dd0 2024-04-27T07:54:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.601+0000 7f8fff7fe640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 0x7f8fe806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.601+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f900011f880 con 0x7f9008144dd0 2024-04-27T07:54:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.601+0000 7f9006ffd640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 0x7f8fe806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.601+0000 7f9006ffd640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 0x7f8fe806a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80012f0 tx=0x7f8ff8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.605+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9000016150 con 0x7f9008144dd0 2024-04-27T07:54:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.765+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f900806be90 con 0x7f8fe8068380 2024-04-27T07:54:02.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.769+0000 7f8fff7fe640 1 -- 172.21.15.22:0/1935046897 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13480 (secure 0 0 0) 0x7f900806be90 con 0x7f8fe8068380 2024-04-27T07:54:02.772 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:02.773 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 msgr2=0x7f8fe806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 0x7f8fe806a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80012f0 tx=0x7f8ff8009040 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 msgr2=0x7f900815f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f900001a190 tx=0x7f9000019300 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 shutdown_connections 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8fe8068380 0x7f8fe806a840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008144dd0 0x7f900815f6b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90080cfc40 0x7f900814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 --2- 172.21.15.22:0/1935046897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900806f870 0x7f90080af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 >> 172.21.15.22:0/1935046897 conn(0x7f9008067f10 msgr2=0x7f900813f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 shutdown_connections 2024-04-27T07:54:02.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.773+0000 7f900d30b640 1 -- 172.21.15.22:0/1935046897 wait complete. 2024-04-27T07:54:02.792 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-04-27T07:54:02.217038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241432,"kb_used_data":4448,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733288,"statfs":{"total":773094113280,"available":772846886912,"internally_reserved":0,"allocated":4554752,"data_stored":2503065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001775"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-27T07:53:57.015565+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:53:57.015565+0000","last_peered":"2024-04-27T07:53:57.015565+0000","last_clean":"2024-04-27T07:53:57.015565+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:53:57.015565+0000","last_undegraded":"2024-04-27T07:53:57.015565+0000","last_fullsized":"2024-04-27T07:53:57.015565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":740295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:54:02.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:54:02.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.913+0000 7fdca8522640 1 Processor -- start 2024-04-27T07:54:02.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.913+0000 7fdca8522640 1 -- start start 2024-04-27T07:54:02.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca8522640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca8522640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 0x7fdca006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca8522640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca0145220 con 0x7fdca006ef10 2024-04-27T07:54:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca8522640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdca0162df0 con 0x7fdca006f560 2024-04-27T07:54:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca8522640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdca01630d0 con 0x7fdca0144e40 2024-04-27T07:54:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca6abf640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 0x7fdca006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:47302/0 (socket says 172.21.15.22:47302) 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 -- 172.21.15.22:0/1238137209 learned_addr learned my addr 172.21.15.22:0/1238137209 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 -- 172.21.15.22:0/1238137209 >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 msgr2=0x7fdca006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 --2- 172.21.15.22:0/1238137209 >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 0x7fdca006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 -- 172.21.15.22:0/1238137209 >> v1:172.21.15.183:6789/0 conn(0x7fdca006ef10 legacy=0x7fdca01623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca62be640 1 -- 172.21.15.22:0/1238137209 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca006e1d0 con 0x7fdca0144e40 2024-04-27T07:54:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.917+0000 7fdca6abf640 1 --2- 172.21.15.22:0/1238137209 >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 0x7fdca006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:54:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca62be640 1 --2- 172.21.15.22:0/1238137209 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009870 tx=0x7fdc9c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9365789b8836501 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca52bc640 1 -- 172.21.15.22:0/1238137209 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a7070 con 0x7fdca0144e40 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca52bc640 1 -- 172.21.15.22:0/1238137209 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c0b0070 con 0x7fdca0144e40 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca52bc640 1 -- 172.21.15.22:0/1238137209 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0ab890 con 0x7fdca0144e40 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- 172.21.15.22:0/1238137209 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 msgr2=0x7fdca006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- 172.21.15.22:0/1238137209 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009870 tx=0x7fdc9c01c560 comp rx=0 tx=0).stop 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- 172.21.15.22:0/1238137209 shutdown_connections 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- 172.21.15.22:0/1238137209 >> v2:172.21.15.183:3300/0 conn(0x7fdca006f560 0x7fdca006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- 172.21.15.22:0/1238137209 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca006e9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- 172.21.15.22:0/1238137209 >> 172.21.15.22:0/1238137209 conn(0x7fdca0067f10 msgr2=0x7fdca006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- 172.21.15.22:0/1238137209 shutdown_connections 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- 172.21.15.22:0/1238137209 wait complete. 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 Processor -- start 2024-04-27T07:54:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- start start 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 0x7fdca01439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca013b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdca00af5e0 con 0x7fdca00b1690 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdca00af460 con 0x7fdca006f560 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca8522640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdca0141010 con 0x7fdca0144e40 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57706/0 (socket says 172.21.15.22:57706) 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 -- 172.21.15.22:0/417626001 learned_addr learned my addr 172.21.15.22:0/417626001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca62be640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 0x7fdca01439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca5abd640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca013b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 -- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 msgr2=0x7fdca013b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca013b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 -- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 msgr2=0x7fdca01439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 0x7fdca01439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca6abf640 1 -- 172.21.15.22:0/417626001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca0143ef0 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.921+0000 7fdca62be640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 0x7fdca01439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca6abf640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800efc0 tx=0x7fdc98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98018030 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9800cde0 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca00b0f80 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98003bc0 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca00b09c0 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca0068c30 con 0x7fdca00b1690 2024-04-27T07:54:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdc9801b070 con 0x7fdca00b1690 2024-04-27T07:54:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 0x7fdc8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fdc980878e0 con 0x7fdca00b1690 2024-04-27T07:54:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca62be640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 0x7fdc8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.925+0000 7fdca62be640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 0x7fdc8406a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0af000 tx=0x7fdc9c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:02.929+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc98022070 con 0x7fdca00b1690 2024-04-27T07:54:03.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.097+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca015f470 con 0x7fdca00b1690 2024-04-27T07:54:03.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.097+0000 7fdc96ffd640 1 -- 172.21.15.22:0/417626001 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+12996 (secure 0 0 0) 0x7fdc98010040 con 0x7fdca00b1690 2024-04-27T07:54:03.102 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:03.102 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"fsid":"807b2bc3-0b23-4239-8346-8619bcda9f4e","created":"2024-04-27T07:50:31.346389+0000","modified":"2024-04-27T07:54:00.018999+0000","last_up_change":"2024-04-27T07:50:41.663816+0000","last_in_change":"2024-04-27T07:50:33.601972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:50:42.186099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3907e5f3-d06c-49c1-a198-583478c871ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6801","nonce":2532053972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6803","nonce":2532053972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6807","nonce":2532053972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2532053972},{"type":"v1","addr":"172.21.15.22:6805","nonce":2532053972}]},"public_addr":"172.21.15.22:6801/2532053972","cluster_addr":"172.21.15.22:6803/2532053972","heartbeat_back_addr":"172.21.15.22:6807/2532053972","heartbeat_front_addr":"172.21.15.22:6805/2532053972","state":["exists","up"]},{"osd":1,"uuid":"902b6447-8080-487a-b5fb-12222688f566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6819","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6821","nonce":73795669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6826","nonce":73795669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6831","nonce":73795669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":73795669},{"type":"v1","addr":"172.21.15.22:6829","nonce":73795669}]},"public_addr":"172.21.15.22:6821/73795669","cluster_addr":"172.21.15.22:6826/73795669","heartbeat_back_addr":"172.21.15.22:6831/73795669","heartbeat_front_addr":"172.21.15.22:6829/73795669","state":["exists","up"]},{"osd":2,"uuid":"c853f889-92ce-4a6f-8f89-475e7ccb18b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6809","nonce":2159087863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6811","nonce":2159087863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6815","nonce":2159087863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2159087863},{"type":"v1","addr":"172.21.15.22:6813","nonce":2159087863}]},"public_addr":"172.21.15.22:6809/2159087863","cluster_addr":"172.21.15.22:6811/2159087863","heartbeat_back_addr":"172.21.15.22:6815/2159087863","heartbeat_front_addr":"172.21.15.22:6813/2159087863","state":["exists","up"]},{"osd":3,"uuid":"91b7cfd1-6847-407c-b6be-6be0e93bbd83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6817","nonce":427337427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6820","nonce":427337427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6825","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6827","nonce":427337427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":427337427},{"type":"v1","addr":"172.21.15.22:6824","nonce":427337427}]},"public_addr":"172.21.15.22:6817/427337427","cluster_addr":"172.21.15.22:6820/427337427","heartbeat_back_addr":"172.21.15.22:6827/427337427","heartbeat_front_addr":"172.21.15.22:6824/427337427","state":["exists","up"]},{"osd":4,"uuid":"dee730ac-0187-42fd-a626-0bf08ea54b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6801","nonce":1968327270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6803","nonce":1968327270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6807","nonce":1968327270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1968327270},{"type":"v1","addr":"172.21.15.183:6805","nonce":1968327270}]},"public_addr":"172.21.15.183:6801/1968327270","cluster_addr":"172.21.15.183:6803/1968327270","heartbeat_back_addr":"172.21.15.183:6807/1968327270","heartbeat_front_addr":"172.21.15.183:6805/1968327270","state":["exists","up"]},{"osd":5,"uuid":"89452ade-5696-4d0b-98e0-af57d4f928e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6817","nonce":2484095365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6819","nonce":2484095365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6823","nonce":2484095365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2484095365},{"type":"v1","addr":"172.21.15.183:6821","nonce":2484095365}]},"public_addr":"172.21.15.183:6817/2484095365","cluster_addr":"172.21.15.183:6819/2484095365","heartbeat_back_addr":"172.21.15.183:6823/2484095365","heartbeat_front_addr":"172.21.15.183:6821/2484095365","state":["exists","up"]},{"osd":6,"uuid":"ea39fb67-4673-40e4-b2b1-8866590cdef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6809","nonce":3842666624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6811","nonce":3842666624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6815","nonce":3842666624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3842666624},{"type":"v1","addr":"172.21.15.183:6813","nonce":3842666624}]},"public_addr":"172.21.15.183:6809/3842666624","cluster_addr":"172.21.15.183:6811/3842666624","heartbeat_back_addr":"172.21.15.183:6815/3842666624","heartbeat_front_addr":"172.21.15.183:6813/3842666624","state":["exists","up"]},{"osd":7,"uuid":"9f9f8998-7b99-4dd6-9637-f1b273f62fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6825","nonce":1208358952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6827","nonce":1208358952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6831","nonce":1208358952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1208358952},{"type":"v1","addr":"172.21.15.183:6829","nonce":1208358952}]},"public_addr":"172.21.15.183:6825/1208358952","cluster_addr":"172.21.15.183:6827/1208358952","heartbeat_back_addr":"172.21.15.183:6831/1208358952","heartbeat_front_addr":"172.21.15.183:6829/1208358952","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.134497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.123340+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.780321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.073214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:40.735503+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.031335+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.272363+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:50:39.218838+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6834/3341477719":"2024-04-28T07:53:57.003848+0000","172.21.15.22:6832/3341477719":"2024-04-28T07:53:57.003848+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:54:03.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 msgr2=0x7fdc8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:03.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 0x7fdc8406a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0af000 tx=0x7fdc9c0a7000 comp rx=0 tx=0).stop 2024-04-27T07:54:03.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 msgr2=0x7fdca00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:03.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800efc0 tx=0x7fdc98009f90 comp rx=0 tx=0).stop 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 shutdown_connections 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fdc84068380 0x7fdc8406a840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca00b1690 0x7fdca00b1b40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdca0144e40 0x7fdca013b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 --2- 172.21.15.22:0/417626001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdca006f560 0x7fdca01439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 >> 172.21.15.22:0/417626001 conn(0x7fdca0067f10 msgr2=0x7fdca00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 shutdown_connections 2024-04-27T07:54:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:03.101+0000 7fdca8522640 1 -- 172.21.15.22:0/417626001 wait complete. 2024-04-27T07:54:04.122 INFO:tasks.ceph:Scrubbing osd.0 2024-04-27T07:54:04.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:04.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.257+0000 7f007167f640 1 Processor -- start 2024-04-27T07:54:04.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.257+0000 7f007167f640 1 -- start start 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f007167f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f006c071f70 0x7f006c072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f007167f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f006c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f007167f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006c072890 con 0x7f006c06ec80 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f007167f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f006c0ce0d0 con 0x7f006c071f70 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f007167f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f006c0cd8c0 con 0x7f006c0cffe0 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f006c06ec80 0x7f006c06f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50624/0 (socket says 172.21.15.22:50624) 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 -- 172.21.15.22:0/3876009937 learned_addr learned my addr 172.21.15.22:0/3876009937 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006a7fc640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f006c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006affd640 1 --2- 172.21.15.22:0/3876009937 >> v2:172.21.15.183:3300/0 conn(0x7f006c071f70 0x7f006c072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645880767 0 0) 0x7f006c072890 con 0x7f006c06ec80 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0054003530 con 0x7f006c06ec80 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3259375334 0 0) 0x7f0054003530 con 0x7f006c06ec80 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 msgr2=0x7f006c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f006c06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 >> v2:172.21.15.183:3300/0 conn(0x7f006c071f70 msgr2=0x7f006c072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 --2- 172.21.15.22:0/3876009937 >> v2:172.21.15.183:3300/0 conn(0x7f006c071f70 0x7f006c072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c0cdb10 con 0x7f006c06ec80 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f0064001a00 con 0x7f006c06ec80 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 >> v1:172.21.15.22:6789/0 conn(0x7f006c06ec80 legacy=0x7f006c06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f00540039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0054003f40 0x7f0054004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0054004d50 0x7f00540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0ce0d0 con 0x7f0054004d50 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0cd8c0 con 0x7f0054003f40 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f005400ba20 con 0x7f006c0cffe0 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0054004d50 0x7f00540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006affd640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f00540039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006a7fc640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0054003f40 0x7f0054004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 -- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 msgr2=0x7f00540039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0cffe0 0x7f00540039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 -- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0054003f40 msgr2=0x7f0054004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0054003f40 0x7f0054004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.261+0000 7f006b7fe640 1 -- 172.21.15.22:0/3876009937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0054003530 con 0x7f0054004d50 2024-04-27T07:54:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006b7fe640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0054004d50 0x7f00540051d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f006400a8d0 tx=0x7f006401fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f543fa4eaec0958e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006400a4c0 con 0x7f0054004d50 2024-04-27T07:54:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00640ad020 con 0x7f0054004d50 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f0069ffb640 1 -- 172.21.15.22:0/3876009937 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a9740 con 0x7f0054004d50 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- 172.21.15.22:0/3876009937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0054004d50 msgr2=0x7f00540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 --2- 172.21.15.22:0/3876009937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0054004d50 0x7f00540051d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f006400a8d0 tx=0x7f006401fb20 comp rx=0 tx=0).stop 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006b7fe640 1 -- 172.21.15.22:0/3876009937 reap_dead start 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- 172.21.15.22:0/3876009937 shutdown_connections 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- 172.21.15.22:0/3876009937 >> 172.21.15.22:0/3876009937 conn(0x7f006c0b21d0 msgr2=0x7f006c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- 172.21.15.22:0/3876009937 shutdown_connections 2024-04-27T07:54:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- 172.21.15.22:0/3876009937 wait complete. 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 Processor -- start 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- start start 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f006c1f2340 0x7f006c134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 0x7f006c0b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0b16d0 0x7f006c137800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f006c06e220 con 0x7f006c1f2720 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f006c06dfd0 con 0x7f006c1f2340 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f007167f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f006c146470 con 0x7f006c0b16d0 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 0x7f006c0b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 0x7f006c0b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57720/0 (socket says 172.21.15.22:57720) 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 -- 172.21.15.22:0/404479169 learned_addr learned my addr 172.21.15.22:0/404479169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006affd640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f006c1f2340 0x7f006c134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006b7fe640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0b16d0 0x7f006c137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 -- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0b16d0 msgr2=0x7f006c137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f006c0b16d0 0x7f006c137800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 -- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f006c1f2340 msgr2=0x7f006c134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f006c1f2340 0x7f006c134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c1352d0 con 0x7f006c1f2720 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.265+0000 7f006a7fc640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 0x7f006c0b1190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00600115e0 tx=0x7f0060011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000f8f0 con 0x7f006c1f2720 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006000fa90 con 0x7f006c1f2720 2024-04-27T07:54:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000fe40 con 0x7f006c1f2720 2024-04-27T07:54:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c1f64b0 con 0x7f006c1f2720 2024-04-27T07:54:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1f6a70 con 0x7f006c1f2720 2024-04-27T07:54:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f006c04ff10 con 0x7f006c1f2720 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0060015460 con 0x7f006c1f2720 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0058068380 0x7f005806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0060087190 con 0x7f006c1f2720 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7f005806e240 0x7f0058070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0060008eb0 con 0x7f005806e240 2024-04-27T07:54:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0060087620 con 0x7f006c1f2720 2024-04-27T07:54:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f006b7fe640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7f005806e240 0x7f0058070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f006affd640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0058068380 0x7f005806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f006b7fe640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7f005806e240 0x7f0058070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f006affd640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0058068380 0x7f005806a840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f005c009ab0 tx=0x7f005c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.269+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== osd.0 v2:172.21.15.22:6800/2532053972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0060008eb0 con 0x7f005806e240 2024-04-27T07:54:04.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 --> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f006c072cb0 con 0x7f005806e240 2024-04-27T07:54:04.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f00537fe640 1 -- 172.21.15.22:0/404479169 <== osd.0 v2:172.21.15.22:6800/2532053972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f006c072cb0 con 0x7f005806e240 2024-04-27T07:54:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:04.288 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7f005806e240 msgr2=0x7f0058070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:6800/2532053972,v1:172.21.15.22:6801/2532053972] conn(0x7f005806e240 0x7f0058070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0058068380 msgr2=0x7f005806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f0058068380 0x7f005806a840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f005c009ab0 tx=0x7f005c008040 comp rx=0 tx=0).stop 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 msgr2=0x7f006c0b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 --2- 172.21.15.22:0/404479169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f006c1f2720 0x7f006c0b1190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00600115e0 tx=0x7f0060011aa0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f006b7fe640 1 -- 172.21.15.22:0/404479169 reap_dead start 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 shutdown_connections 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 >> 172.21.15.22:0/404479169 conn(0x7f006c0b21d0 msgr2=0x7f006c145340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 shutdown_connections 2024-04-27T07:54:04.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.285+0000 7f007167f640 1 -- 172.21.15.22:0/404479169 wait complete. 2024-04-27T07:54:04.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-27T07:54:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 Processor -- start 2024-04-27T07:54:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 -- start start 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f515c0d1e70 0x7f515c1696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0d32f0 con 0x7f515c0ba360 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0d35d0 con 0x7f515c15ec10 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5163fd4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f515c15eff0 con 0x7f515c0d1e70 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5162571640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f515c0d1e70 0x7f515c1696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f516156f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f515c0ba360 0x7f515c0d1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50636/0 (socket says 172.21.15.22:50636) 2024-04-27T07:54:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5162571640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f515c0d1e70 0x7f515c1696d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57728/0 (socket says 172.21.15.22:57728) 2024-04-27T07:54:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f516156f640 1 -- 172.21.15.22:0/1978741491 learned_addr learned my addr 172.21.15.22:0/1978741491 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5162571640 1 -- 172.21.15.22:0/1978741491 >> v1:172.21.15.183:6789/0 conn(0x7f515c15ec10 legacy=0x7f515c0d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5162571640 1 -- 172.21.15.22:0/1978741491 >> v1:172.21.15.22:6789/0 conn(0x7f515c0ba360 legacy=0x7f515c0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5162571640 1 -- 172.21.15.22:0/1978741491 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0d38b0 con 0x7f515c0d1e70 2024-04-27T07:54:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.413+0000 7f5160d6e640 1 -- 172.21.15.22:0/1978741491 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915544248 0 0) 0x7f515c0d32f0 con 0x7f515c0ba360 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5162571640 1 --2- 172.21.15.22:0/1978741491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c1696d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5154009d90 tx=0x7f515401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf31b0f351ed25e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5160d6e640 1 -- 172.21.15.22:0/1978741491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540b2070 con 0x7f515c0d1e70 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5160d6e640 1 -- 172.21.15.22:0/1978741491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51540a5830 con 0x7f515c0d1e70 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5160d6e640 1 -- 172.21.15.22:0/1978741491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540a5c00 con 0x7f515c0d1e70 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 -- 172.21.15.22:0/1978741491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 msgr2=0x7f515c1696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 --2- 172.21.15.22:0/1978741491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c1696d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5154009d90 tx=0x7f515401c970 comp rx=0 tx=0).stop 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 -- 172.21.15.22:0/1978741491 shutdown_connections 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 --2- 172.21.15.22:0/1978741491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c1696d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 -- 172.21.15.22:0/1978741491 >> 172.21.15.22:0/1978741491 conn(0x7f515c067d10 msgr2=0x7f515c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 -- 172.21.15.22:0/1978741491 shutdown_connections 2024-04-27T07:54:04.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.417+0000 7f5163fd4640 1 -- 172.21.15.22:0/1978741491 wait complete. 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 Processor -- start 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- start start 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c0ba360 0x7f515c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f515c15ec10 0x7f515c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f515c16c5b0 con 0x7f515c0d1e70 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f515c16c430 con 0x7f515c0ba360 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f515c16c730 con 0x7f515c15ec10 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57738/0 (socket says 172.21.15.22:57738) 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 -- 172.21.15.22:0/722681503 learned_addr learned my addr 172.21.15.22:0/722681503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:04.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5162571640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f515c15ec10 0x7f515c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5161d70640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c0ba360 0x7f515c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 -- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f515c15ec10 msgr2=0x7f515c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f515c15ec10 0x7f515c15fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 -- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c0ba360 msgr2=0x7f515c153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c0ba360 0x7f515c153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 -- 172.21.15.22:0/722681503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c149bf0 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f516156f640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c410 tx=0x7f514c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0074e0 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c0047f0 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c149ed0 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c007680 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c14a490 con 0x7f515c0d1e70 2024-04-27T07:54:04.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c14a690 con 0x7f515c0d1e70 2024-04-27T07:54:04.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f514c011070 con 0x7f515c0d1e70 2024-04-27T07:54:04.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f514c086270 con 0x7f515c0d1e70 2024-04-27T07:54:04.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.421+0000 7f5161d70640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.425+0000 7f5161d70640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 0x7f514006a840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5158000f70 tx=0x7f5158009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.429+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f514c056b30 con 0x7f515c0d1e70 2024-04-27T07:54:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.585+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f515c0b6b40 con 0x7f5140068380 2024-04-27T07:54:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f51527fc640 1 -- 172.21.15.22:0/722681503 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f515c0b6b40 con 0x7f5140068380 2024-04-27T07:54:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 0 to deep-scrub 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 msgr2=0x7f514006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 0x7f514006a840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5158000f70 tx=0x7f5158009040 comp rx=0 tx=0).stop 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 msgr2=0x7f515c161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c410 tx=0x7f514c00c8d0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 shutdown_connections 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f515c15ec10 0x7f515c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515c0d1e70 0x7f515c161840 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 --2- 172.21.15.22:0/722681503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c0ba360 0x7f515c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 >> 172.21.15.22:0/722681503 conn(0x7f515c067d10 msgr2=0x7f515c06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 shutdown_connections 2024-04-27T07:54:04.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.589+0000 7f5163fd4640 1 -- 172.21.15.22:0/722681503 wait complete. 2024-04-27T07:54:04.609 INFO:tasks.ceph:Scrubbing osd.1 2024-04-27T07:54:04.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:04.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.729+0000 7f5f5e0e2640 1 Processor -- start 2024-04-27T07:54:04.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.729+0000 7f5f5e0e2640 1 -- start start 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5e0e2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f5809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5e0e2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f580afa50 con 0x7f5f58073400 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5e0e2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5815d860 con 0x7f5f5806c2e0 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5e0e2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5f580b3880 con 0x7f5f5815c510 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5c8df640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5f58073400 0x7f5f5806bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50646/0 (socket says 172.21.15.22:50646) 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5c8df640 1 -- 172.21.15.22:0/1289682977 learned_addr learned my addr 172.21.15.22:0/1289682977 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f5809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877338791 0 0) 0x7f5f580afa50 con 0x7f5f58073400 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f44003530 con 0x7f5f58073400 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2625691091 0 0) 0x7f5f44003530 con 0x7f5f58073400 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 msgr2=0x7f5f5809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f5809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 >> v1:172.21.15.183:6789/0 conn(0x7f5f5806c2e0 legacy=0x7f5f581600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f580b3b60 con 0x7f5f58073400 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f5f48001680 con 0x7f5f58073400 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 >> v1:172.21.15.22:6789/0 conn(0x7f5f58073400 legacy=0x7f5f5806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f440039f0 0x7f5f44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f44004d50 0x7f5f440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580b3880 con 0x7f5f440039f0 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4400ba80 con 0x7f5f44004d50 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4400bc80 con 0x7f5f5815c510 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f440039f0 0x7f5f44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d8e1640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f44004d50 0x7f5f440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5c8df640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 -- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 msgr2=0x7f5f44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 -- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f44004d50 msgr2=0x7f5f440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f44004d50 0x7f5f440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5d0e0640 1 -- 172.21.15.22:0/1289682977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f44003530 con 0x7f5f440039f0 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.733+0000 7f5f5c8df640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f5815c510 0x7f5f44004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f440039f0 0x7f5f44003e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009ab0 tx=0x7f5f5401d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f896f4771e92b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f540b2070 con 0x7f5f440039f0 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f540a5530 con 0x7f5f440039f0 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4ffff640 1 -- 172.21.15.22:0/1289682977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f540096b0 con 0x7f5f440039f0 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1289682977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f440039f0 msgr2=0x7f5f44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 --2- 172.21.15.22:0/1289682977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f440039f0 0x7f5f44003e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009ab0 tx=0x7f5f5401d4b0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1289682977 reap_dead start 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1289682977 shutdown_connections 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1289682977 >> 172.21.15.22:0/1289682977 conn(0x7f5f580b6a10 msgr2=0x7f5f580b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1289682977 shutdown_connections 2024-04-27T07:54:04.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1289682977 wait complete. 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 Processor -- start 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- start start 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f440039f0 0x7f5f58145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f580b4640 0x7f5f580b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 0x7f5f5809e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58168490 con 0x7f5f5809de30 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5809ae40 con 0x7f5f580b4640 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5f58168610 con 0x7f5f440039f0 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 0x7f5f5809e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 0x7f5f5809e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57752/0 (socket says 172.21.15.22:57752) 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1261663867 learned_addr learned my addr 172.21.15.22:0/1261663867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f440039f0 0x7f5f58145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5c8df640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f580b4640 0x7f5f580b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f440039f0 msgr2=0x7f5f58145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f440039f0 0x7f5f58145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f580b4640 msgr2=0x7f5f580b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f580b4640 0x7f5f580b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5809b6a0 con 0x7f5f5809de30 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5d8e1640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 0x7f5f5809e2b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5f5806e850 tx=0x7f5f5000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50003b90 con 0x7f5f5809de30 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f50018070 con 0x7f5f5809de30 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5809e7f0 con 0x7f5f5809de30 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500050d0 con 0x7f5f5809de30 2024-04-27T07:54:04.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.737+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5809ed80 con 0x7f5f5809de30 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f5804fea0 con 0x7f5f5809de30 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5f50005270 con 0x7f5f5809de30 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f3c0683b0 0x7f5f3c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f5f50086d50 con 0x7f5f5809de30 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7f5f3c06e270 0x7f5f3c070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f5000bf30 con 0x7f5f3c06e270 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5f500871e0 con 0x7f5f5809de30 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f5c8df640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7f5f3c06e270 0x7f5f3c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f3c0683b0 0x7f5f3c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f5c8df640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7f5f3c06e270 0x7f5f3c070690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f5d0e0640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f3c0683b0 0x7f5f3c06a870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f540af000 tx=0x7f5f540a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.741+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== osd.1 v2:172.21.15.22:6819/73795669 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f5000bf30 con 0x7f5f3c06e270 2024-04-27T07:54:04.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.753+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 --> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5f580cd770 con 0x7f5f3c06e270 2024-04-27T07:54:04.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f4dffb640 1 -- 172.21.15.22:0/1261663867 <== osd.1 v2:172.21.15.22:6819/73795669 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5f580cd770 con 0x7f5f3c06e270 2024-04-27T07:54:04.759 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:04.759 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:04.759 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7f5f3c06e270 msgr2=0x7f5f3c070690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:6819/73795669,v1:172.21.15.22:6821/73795669] conn(0x7f5f3c06e270 0x7f5f3c070690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f3c0683b0 msgr2=0x7f5f3c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f5f3c0683b0 0x7f5f3c06a870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f540af000 tx=0x7f5f540a7040 comp rx=0 tx=0).stop 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 msgr2=0x7f5f5809e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 --2- 172.21.15.22:0/1261663867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f5809de30 0x7f5f5809e2b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5f5806e850 tx=0x7f5f5000e9e0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5d8e1640 1 -- 172.21.15.22:0/1261663867 reap_dead start 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 shutdown_connections 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 >> 172.21.15.22:0/1261663867 conn(0x7f5f580b6a10 msgr2=0x7f5f58159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 shutdown_connections 2024-04-27T07:54:04.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.757+0000 7f5f5e0e2640 1 -- 172.21.15.22:0/1261663867 wait complete. 2024-04-27T07:54:04.773 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-27T07:54:04.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.881+0000 7fdab590c640 1 Processor -- start 2024-04-27T07:54:04.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.881+0000 7fdab590c640 1 -- start start 2024-04-27T07:54:04.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdab590c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdab00b1db0 0x7fdab01602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdab590c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 0x7fdab00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdab590c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab00cd1e0 con 0x7fdab01607e0 2024-04-27T07:54:04.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdab590c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdab00cd4c0 con 0x7fdab00b1db0 2024-04-27T07:54:04.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdab590c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdab00cd7a0 con 0x7fdab015c8e0 2024-04-27T07:54:04.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdab00b1db0 0x7fdab01602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 0x7fdab00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaae7fc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fdab01607e0 0x7fdab015c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:42706/0 (socket says 172.21.15.22:42706) 2024-04-27T07:54:04.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdab00b1db0 0x7fdab01602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.22:51324/0 (socket says 172.21.15.22:51324) 2024-04-27T07:54:04.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaae7fc640 1 -- 172.21.15.22:0/2759697750 learned_addr learned my addr 172.21.15.22:0/2759697750 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:04.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 0x7fdab00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:47386/0 (socket says 172.21.15.22:47386) 2024-04-27T07:54:04.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 -- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 msgr2=0x7fdab00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:04.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 --2- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 0x7fdab00b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 -- 172.21.15.22:0/2759697750 >> v1:172.21.15.183:6789/0 conn(0x7fdab01607e0 legacy=0x7fdab015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 -- 172.21.15.22:0/2759697750 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab00cda80 con 0x7fdab00b1db0 2024-04-27T07:54:04.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.885+0000 7fdaaeffd640 1 --2- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab00b1db0 0x7fdab01602a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00097c0 tx=0x7fdaa001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5de1456169da0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaadffb640 1 -- 172.21.15.22:0/2759697750 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00b0070 con 0x7fdab00b1db0 2024-04-27T07:54:04.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaadffb640 1 -- 172.21.15.22:0/2759697750 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa00ab600 con 0x7fdab00b1db0 2024-04-27T07:54:04.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaadffb640 1 -- 172.21.15.22:0/2759697750 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00ab9d0 con 0x7fdab00b1db0 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab00b1db0 msgr2=0x7fdab01602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab00b1db0 0x7fdab01602a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00097c0 tx=0x7fdaa001c410 comp rx=0 tx=0).stop 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- 172.21.15.22:0/2759697750 shutdown_connections 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab015c8e0 0x7fdab00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- 172.21.15.22:0/2759697750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab00b1db0 0x7fdab01602a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- 172.21.15.22:0/2759697750 >> 172.21.15.22:0/2759697750 conn(0x7fdab00d2580 msgr2=0x7fdab00d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- 172.21.15.22:0/2759697750 shutdown_connections 2024-04-27T07:54:04.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- 172.21.15.22:0/2759697750 wait complete. 2024-04-27T07:54:04.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 Processor -- start 2024-04-27T07:54:04.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- start start 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab00b1db0 0x7fdab0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 0x7fdab00d0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00cebc0 con 0x7fdab015c8e0 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00cea40 con 0x7fdab01607e0 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdab590c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdab00ced40 con 0x7fdab00b1db0 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57766/0 (socket says 172.21.15.22:57766) 2024-04-27T07:54:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 -- 172.21.15.22:0/2706733507 learned_addr learned my addr 172.21.15.22:0/2706733507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaaeffd640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab00b1db0 0x7fdab0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaaf7fe640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 0x7fdab00d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 -- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab00b1db0 msgr2=0x7fdab0154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab00b1db0 0x7fdab0154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 -- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 msgr2=0x7fdab00d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 0x7fdab00d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 -- 172.21.15.22:0/2706733507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01f62f0 con 0x7fdab015c8e0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaaf7fe640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 0x7fdab00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fdaae7fc640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c370 tx=0x7fdaa400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.889+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4017070 con 0x7fdab015c8e0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa4005cf0 con 0x7fdab015c8e0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab00ca2c0 con 0x7fdab015c8e0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40072f0 con 0x7fdab015c8e0 2024-04-27T07:54:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab00ca880 con 0x7fdab015c8e0 2024-04-27T07:54:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab0069340 con 0x7fdab015c8e0 2024-04-27T07:54:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdaa401a070 con 0x7fdab015c8e0 2024-04-27T07:54:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fda97fff640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 0x7fda9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fdaa40859a0 con 0x7fdab015c8e0 2024-04-27T07:54:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fdaaeffd640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 0x7fda9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.893+0000 7fdaaeffd640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 0x7fda9c06a840 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00ad000 tx=0x7fdaa00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:04.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:04.897+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdaa4055c10 con 0x7fdab015c8e0 2024-04-27T07:54:05.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.061+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fdab00b33e0 con 0x7fda9c068380 2024-04-27T07:54:05.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fda97fff640 1 -- 172.21.15.22:0/2706733507 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdab00b33e0 con 0x7fda9c068380 2024-04-27T07:54:05.068 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 1 to deep-scrub 2024-04-27T07:54:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 msgr2=0x7fda9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 0x7fda9c06a840 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00ad000 tx=0x7fdaa00a6000 comp rx=0 tx=0).stop 2024-04-27T07:54:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 msgr2=0x7fdab00d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c370 tx=0x7fdaa400c830 comp rx=0 tx=0).stop 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 shutdown_connections 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fda9c068380 0x7fda9c06a840 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab01607e0 0x7fdab00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdab015c8e0 0x7fdab00d0130 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 --2- 172.21.15.22:0/2706733507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdab00b1db0 0x7fdab0154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 >> 172.21.15.22:0/2706733507 conn(0x7fdab00d2580 msgr2=0x7fdab00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 shutdown_connections 2024-04-27T07:54:05.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.065+0000 7fdab590c640 1 -- 172.21.15.22:0/2706733507 wait complete. 2024-04-27T07:54:05.088 INFO:tasks.ceph:Scrubbing osd.2 2024-04-27T07:54:05.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:05.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.213+0000 7f83642c8640 1 Processor -- start 2024-04-27T07:54:05.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.213+0000 7f83642c8640 1 -- start start 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 0x7f835c072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f835c06ee50 0x7f835c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c072890 con 0x7f835c0cffe0 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0ce0d0 con 0x7f835c06ee50 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0cd8c0 con 0x7f835c071f70 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f835c06ee50 0x7f835c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f835c06ee50 0x7f835c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57768/0 (socket says 172.21.15.22:57768) 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 -- 172.21.15.22:0/3019545277 learned_addr learned my addr 172.21.15.22:0/3019545277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362064640 1 --2- 172.21.15.22:0/3019545277 >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 0x7f835c072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 -- 172.21.15.22:0/3019545277 >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 msgr2=0x7f835c072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:05.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 --2- 172.21.15.22:0/3019545277 >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 0x7f835c072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 -- 172.21.15.22:0/3019545277 >> v1:172.21.15.183:6789/0 conn(0x7f835c0cffe0 legacy=0x7f835c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 -- 172.21.15.22:0/3019545277 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0cdb10 con 0x7f835c06ee50 2024-04-27T07:54:05.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362064640 1 --2- 172.21.15.22:0/3019545277 >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 0x7f835c072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8362865640 1 --2- 172.21.15.22:0/3019545277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c06ee50 0x7f835c06f280 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f835c06cf00 tx=0x7f835401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272fb298bc2e5513 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8361062640 1 -- 172.21.15.22:0/3019545277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540a5620 con 0x7f835c06ee50 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8361062640 1 -- 172.21.15.22:0/3019545277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83540a57c0 con 0x7f835c06ee50 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f8361062640 1 -- 172.21.15.22:0/3019545277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540a5b90 con 0x7f835c06ee50 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- 172.21.15.22:0/3019545277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c06ee50 msgr2=0x7f835c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 --2- 172.21.15.22:0/3019545277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c06ee50 0x7f835c06f280 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f835c06cf00 tx=0x7f835401c960 comp rx=0 tx=0).stop 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- 172.21.15.22:0/3019545277 shutdown_connections 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 --2- 172.21.15.22:0/3019545277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c06ee50 0x7f835c06f280 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 --2- 172.21.15.22:0/3019545277 >> v2:172.21.15.183:3300/0 conn(0x7f835c071f70 0x7f835c072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- 172.21.15.22:0/3019545277 >> 172.21.15.22:0/3019545277 conn(0x7f835c0b21d0 msgr2=0x7f835c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.217+0000 7f83642c8640 1 -- 172.21.15.22:0/3019545277 shutdown_connections 2024-04-27T07:54:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- 172.21.15.22:0/3019545277 wait complete. 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 Processor -- start 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- start start 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c06ee50 0x7f835c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f835c071f70 0x7f835c1f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 0x7f835c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f835c073130 con 0x7f835c0cffe0 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f835c072fb0 con 0x7f835c06ee50 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f835c0cceb0 con 0x7f835c071f70 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 0x7f835c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 0x7f835c0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57772/0 (socket says 172.21.15.22:57772) 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 -- 172.21.15.22:0/1868206751 learned_addr learned my addr 172.21.15.22:0/1868206751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362064640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c06ee50 0x7f835c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8361863640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f835c071f70 0x7f835c1f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 -- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f835c071f70 msgr2=0x7f835c1f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f835c071f70 0x7f835c1f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 -- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c06ee50 msgr2=0x7f835c15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c06ee50 0x7f835c15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1f8a00 con 0x7f835c0cffe0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8362865640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 0x7f835c0add00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8354019570 tx=0x7f8354019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540bc020 con 0x7f835c0cffe0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83540b8670 con 0x7f835c0cffe0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c1f8fc0 con 0x7f835c0cffe0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354009510 con 0x7f835c0cffe0 2024-04-27T07:54:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c1f9690 con 0x7f835c0cffe0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.221+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f835c04ff10 con 0x7f835c0cffe0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83540bf020 con 0x7f835c0cffe0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8340068380 0x7f834006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f8354121560 con 0x7f835c0cffe0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f834006e240 0x7f8340070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8340070d10 con 0x7f834006e240 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8354128050 con 0x7f835c0cffe0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8362064640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8340068380 0x7f834006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8361863640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f834006e240 0x7f8340070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8361863640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f834006e240 0x7f8340070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8362064640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8340068380 0x7f834006a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8358009950 tx=0x7f8358008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.225+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== osd.2 v2:172.21.15.22:6808/2159087863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8340070d10 con 0x7f834006e240 2024-04-27T07:54:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.237+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 --> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f835c072cb0 con 0x7f834006e240 2024-04-27T07:54:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.237+0000 7f8352ffd640 1 -- 172.21.15.22:0/1868206751 <== osd.2 v2:172.21.15.22:6808/2159087863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f835c072cb0 con 0x7f834006e240 2024-04-27T07:54:05.243 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:05.243 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:05.243 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f834006e240 msgr2=0x7f8340070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:6808/2159087863,v1:172.21.15.22:6809/2159087863] conn(0x7f834006e240 0x7f8340070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8340068380 msgr2=0x7f834006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f8340068380 0x7f834006a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8358009950 tx=0x7f8358008040 comp rx=0 tx=0).stop 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 msgr2=0x7f835c0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 --2- 172.21.15.22:0/1868206751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c0cffe0 0x7f835c0add00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8354019570 tx=0x7f8354019700 comp rx=0 tx=0).stop 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f8362865640 1 -- 172.21.15.22:0/1868206751 reap_dead start 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 shutdown_connections 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 >> 172.21.15.22:0/1868206751 conn(0x7f835c0b21d0 msgr2=0x7f835c145490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 shutdown_connections 2024-04-27T07:54:05.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.241+0000 7f83642c8640 1 -- 172.21.15.22:0/1868206751 wait complete. 2024-04-27T07:54:05.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-27T07:54:05.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 Processor -- start 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 -- start start 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f91fc15eba0 0x7f91fc15ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc168620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91fc06e940 con 0x7f91fc0b3e00 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f91fc15f650 con 0x7f91fc15eba0 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9202807640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc15f930 con 0x7f91fc06d2f0 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f91fc15eba0 0x7f91fc15ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f91fc15eba0 0x7f91fc15ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57774/0 (socket says 172.21.15.22:57774) 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 -- 172.21.15.22:0/1456555193 learned_addr learned my addr 172.21.15.22:0/1456555193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f9200da4640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 -- 172.21.15.22:0/1456555193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 msgr2=0x7f91fc168620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc168620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 -- 172.21.15.22:0/1456555193 >> v1:172.21.15.183:6789/0 conn(0x7f91fc0b3e00 legacy=0x7f91fc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.377+0000 7f91fbfff640 1 -- 172.21.15.22:0/1456555193 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc15fc10 con 0x7f91fc15eba0 2024-04-27T07:54:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91fbfff640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc15ef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098a0 tx=0x7f91ec01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f05768fe36554f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91faffd640 1 -- 172.21.15.22:0/1456555193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0aa660 con 0x7f91fc15eba0 2024-04-27T07:54:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91faffd640 1 -- 172.21.15.22:0/1456555193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ec0af070 con 0x7f91fc15eba0 2024-04-27T07:54:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91faffd640 1 -- 172.21.15.22:0/1456555193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0aaa20 con 0x7f91fc15eba0 2024-04-27T07:54:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- 172.21.15.22:0/1456555193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 msgr2=0x7f91fc15ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc15ef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098a0 tx=0x7f91ec01c4e0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- 172.21.15.22:0/1456555193 shutdown_connections 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- 172.21.15.22:0/1456555193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc15ef80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- 172.21.15.22:0/1456555193 >> 172.21.15.22:0/1456555193 conn(0x7f91fc072870 msgr2=0x7f91fc072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- 172.21.15.22:0/1456555193 shutdown_connections 2024-04-27T07:54:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- 172.21.15.22:0/1456555193 wait complete. 2024-04-27T07:54:05.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 Processor -- start 2024-04-27T07:54:05.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- start start 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc0b3e00 0x7f91fc14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc16b710 con 0x7f91fc15eba0 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc16b590 con 0x7f91fc0b3e00 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9202807640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc16b890 con 0x7f91fc06d2f0 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57778/0 (socket says 172.21.15.22:57778) 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 -- 172.21.15.22:0/838384419 learned_addr learned my addr 172.21.15.22:0/838384419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91fbfff640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91fb7fe640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc0b3e00 0x7f91fc14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 -- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 msgr2=0x7f91fc1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc1551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 -- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc0b3e00 msgr2=0x7f91fc14a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc0b3e00 0x7f91fc14a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f9200da4640 1 -- 172.21.15.22:0/838384419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc1558a0 con 0x7f91fc15eba0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.381+0000 7f91fbfff640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f9200da4640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f91f4007e40 tx=0x7f91f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4014620 con 0x7f91fc15eba0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4019070 con 0x7f91fc15eba0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4014a20 con 0x7f91fc15eba0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc155b80 con 0x7f91fc15eba0 2024-04-27T07:54:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc156140 con 0x7f91fc15eba0 2024-04-27T07:54:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc069340 con 0x7f91fc15eba0 2024-04-27T07:54:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f91f4014bc0 con 0x7f91fc15eba0 2024-04-27T07:54:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 0x7f91e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f91f4086fe0 con 0x7f91fc15eba0 2024-04-27T07:54:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91fbfff640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 0x7f91e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.385+0000 7f91fbfff640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 0x7f91e806a870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0ae000 tx=0x7f91ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.389+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91f4057820 con 0x7f91fc15eba0 2024-04-27T07:54:05.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.545+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f91fc0af5a0 con 0x7f91e80683b0 2024-04-27T07:54:05.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.545+0000 7f91f8ff9640 1 -- 172.21.15.22:0/838384419 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f91fc0af5a0 con 0x7f91e80683b0 2024-04-27T07:54:05.550 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 2 to deep-scrub 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 msgr2=0x7f91e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 0x7f91e806a870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0ae000 tx=0x7f91ec0a7000 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 msgr2=0x7f91fc149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f91f4007e40 tx=0x7f91f400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 shutdown_connections 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f91e80683b0 0x7f91e806a870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91fc15eba0 0x7f91fc149820 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc0b3e00 0x7f91fc14a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 --2- 172.21.15.22:0/838384419 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91fc06d2f0 0x7f91fc1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 >> 172.21.15.22:0/838384419 conn(0x7f91fc072870 msgr2=0x7f91fc0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 shutdown_connections 2024-04-27T07:54:05.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.549+0000 7f9202807640 1 -- 172.21.15.22:0/838384419 wait complete. 2024-04-27T07:54:05.568 INFO:tasks.ceph:Scrubbing osd.3 2024-04-27T07:54:05.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:05.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 Processor -- start 2024-04-27T07:54:05.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 -- start start 2024-04-27T07:54:05.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f61500b1f70 0x7f61500b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61500cea80 con 0x7f615009ae10 2024-04-27T07:54:05.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f61500ced60 con 0x7f61500b1f70 2024-04-27T07:54:05.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.697+0000 7f6155a08640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f61500b3700 con 0x7f615009aa30 2024-04-27T07:54:05.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.22:47428/0 (socket says 172.21.15.22:47428) 2024-04-27T07:54:05.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f61500b1f70 0x7f61500b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614effd640 1 -- 172.21.15.22:0/963705059 learned_addr learned my addr 172.21.15.22:0/963705059 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614e7fc640 1 --1- 172.21.15.22:0/963705059 >> v1:172.21.15.183:6789/0 conn(0x7f615009ae10 0x7f615009b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:42738/0 (socket says 172.21.15.22:42738) 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 -- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 msgr2=0x7f61500b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 -- 172.21.15.22:0/963705059 >> v1:172.21.15.183:6789/0 conn(0x7f615009ae10 legacy=0x7f615009b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 -- 172.21.15.22:0/963705059 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61500cdeb0 con 0x7f61500b1f70 2024-04-27T07:54:05.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614effd640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:54:05.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614f7fe640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61500b1f70 0x7f61500b2400 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6148009d90 tx=0x7f614801c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a347fd9509a5270 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614dffb640 1 -- 172.21.15.22:0/963705059 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480b2070 con 0x7f61500b1f70 2024-04-27T07:54:05.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614dffb640 1 -- 172.21.15.22:0/963705059 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61480a5930 con 0x7f61500b1f70 2024-04-27T07:54:05.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f614dffb640 1 -- 172.21.15.22:0/963705059 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480a5d00 con 0x7f61500b1f70 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 -- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61500b1f70 msgr2=0x7f61500b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61500b1f70 0x7f61500b2400 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6148009d90 tx=0x7f614801c850 comp rx=0 tx=0).stop 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 -- 172.21.15.22:0/963705059 shutdown_connections 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61500b1f70 0x7f61500b2400 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 --2- 172.21.15.22:0/963705059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f615009aa30 0x7f61500b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.701+0000 7f6155a08640 1 -- 172.21.15.22:0/963705059 >> 172.21.15.22:0/963705059 conn(0x7f61500b4100 msgr2=0x7f61500b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- 172.21.15.22:0/963705059 shutdown_connections 2024-04-27T07:54:05.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- 172.21.15.22:0/963705059 wait complete. 2024-04-27T07:54:05.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 Processor -- start 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- start start 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f615009aa30 0x7f6150155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 0x7f6150154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61500b1f70 0x7f61500af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61500ce550 con 0x7f615009ae10 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61500ce3d0 con 0x7f615009aa30 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6150169df0 con 0x7f61500b1f70 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 0x7f6150154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 0x7f6150154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57790/0 (socket says 172.21.15.22:57790) 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 -- 172.21.15.22:0/1827665750 learned_addr learned my addr 172.21.15.22:0/1827665750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614effd640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f615009aa30 0x7f6150155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614f7fe640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61500b1f70 0x7f61500af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 -- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61500b1f70 msgr2=0x7f61500af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61500b1f70 0x7f61500af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 -- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f615009aa30 msgr2=0x7f6150155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f615009aa30 0x7f6150155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61501554c0 con 0x7f615009ae10 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f614e7fc640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 0x7f6150154f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f614400c420 tx=0x7f614400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440074e0 con 0x7f615009ae10 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6144007680 con 0x7f615009ae10 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6150166850 con 0x7f615009ae10 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144004a30 con 0x7f615009ae10 2024-04-27T07:54:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615006cf30 con 0x7f615009ae10 2024-04-27T07:54:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.705+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f615004ff10 con 0x7f615009ae10 2024-04-27T07:54:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f614401a020 con 0x7f615009ae10 2024-04-27T07:54:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f613c068380 0x7f613c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6144085f00 con 0x7f615009ae10 2024-04-27T07:54:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f613c06e240 0x7f613c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f613c070d10 con 0x7f613c06e240 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6144088600 con 0x7f615009ae10 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f614effd640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f613c068380 0x7f613c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f614f7fe640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f613c06e240 0x7f613c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f614f7fe640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f613c06e240 0x7f613c070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f614effd640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f613c068380 0x7f613c06a840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6140009af0 tx=0x7f6140008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.709+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== osd.3 v2:172.21.15.22:6816/427337427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f613c070d10 con 0x7f613c06e240 2024-04-27T07:54:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.721+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 --> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f61500cc710 con 0x7f613c06e240 2024-04-27T07:54:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6137fff640 1 -- 172.21.15.22:0/1827665750 <== osd.3 v2:172.21.15.22:6816/427337427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f61500cc710 con 0x7f613c06e240 2024-04-27T07:54:05.727 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:05.727 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f613c06e240 msgr2=0x7f613c070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:6816/427337427,v1:172.21.15.22:6817/427337427] conn(0x7f613c06e240 0x7f613c070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f613c068380 msgr2=0x7f613c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f613c068380 0x7f613c06a840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6140009af0 tx=0x7f6140008040 comp rx=0 tx=0).stop 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 msgr2=0x7f6150154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 --2- 172.21.15.22:0/1827665750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615009ae10 0x7f6150154f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f614400c420 tx=0x7f614400c8e0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f614f7fe640 1 -- 172.21.15.22:0/1827665750 reap_dead start 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 shutdown_connections 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 >> 172.21.15.22:0/1827665750 conn(0x7f61500b4100 msgr2=0x7f61500701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 shutdown_connections 2024-04-27T07:54:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.725+0000 7f6155a08640 1 -- 172.21.15.22:0/1827665750 wait complete. 2024-04-27T07:54:05.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-27T07:54:05.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 Processor -- start 2024-04-27T07:54:05.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 -- start start 2024-04-27T07:54:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f13401454b0 0x7f1340072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1340145a00 0x7f134006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134006e050 con 0x7f1340072030 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f134006f330 con 0x7f1340145a00 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1344eac640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f13400ceae0 con 0x7f13401454b0 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1335d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1340145a00 0x7f134006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f133ed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1340072030 0x7f134015cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50656/0 (socket says 172.21.15.22:50656) 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f133ed9d640 1 -- 172.21.15.22:0/160477575 learned_addr learned my addr 172.21.15.22:0/160477575 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f133e59c640 1 --2- 172.21.15.22:0/160477575 >> v2:172.21.15.183:3300/0 conn(0x7f13401454b0 0x7f1340072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1335d9b640 1 -- 172.21.15.22:0/160477575 >> v2:172.21.15.183:3300/0 conn(0x7f13401454b0 msgr2=0x7f1340072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1335d9b640 1 --2- 172.21.15.22:0/160477575 >> v2:172.21.15.183:3300/0 conn(0x7f13401454b0 0x7f1340072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1335d9b640 1 -- 172.21.15.22:0/160477575 >> v1:172.21.15.22:6789/0 conn(0x7f1340072030 legacy=0x7f134015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.857+0000 7f1335d9b640 1 -- 172.21.15.22:0/160477575 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13400cedc0 con 0x7f1340145a00 2024-04-27T07:54:05.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1335d9b640 1 --2- 172.21.15.22:0/160477575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1340145a00 0x7f134006edf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13401465a0 tx=0x7f132801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a0db5a90c22347 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f133dd9b640 1 -- 172.21.15.22:0/160477575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b0070 con 0x7f1340145a00 2024-04-27T07:54:05.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f133dd9b640 1 -- 172.21.15.22:0/160477575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280ab600 con 0x7f1340145a00 2024-04-27T07:54:05.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f133dd9b640 1 -- 172.21.15.22:0/160477575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280ab9d0 con 0x7f1340145a00 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 -- 172.21.15.22:0/160477575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1340145a00 msgr2=0x7f134006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 --2- 172.21.15.22:0/160477575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1340145a00 0x7f134006edf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13401465a0 tx=0x7f132801c480 comp rx=0 tx=0).stop 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 -- 172.21.15.22:0/160477575 shutdown_connections 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 --2- 172.21.15.22:0/160477575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1340145a00 0x7f134006edf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 --2- 172.21.15.22:0/160477575 >> v2:172.21.15.183:3300/0 conn(0x7f13401454b0 0x7f1340072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 -- 172.21.15.22:0/160477575 >> 172.21.15.22:0/160477575 conn(0x7f1340067f10 msgr2=0x7f134006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 -- 172.21.15.22:0/160477575 shutdown_connections 2024-04-27T07:54:05.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.861+0000 7f1344eac640 1 -- 172.21.15.22:0/160477575 wait complete. 2024-04-27T07:54:05.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 Processor -- start 2024-04-27T07:54:05.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- start start 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1340072030 0x7f134015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1340145a00 0x7f134013e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1340072850 con 0x7f13401454b0 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f134015d820 con 0x7f1340145a00 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f13400b2db0 con 0x7f1340072030 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57816/0 (socket says 172.21.15.22:57816) 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 -- 172.21.15.22:0/4214446786 learned_addr learned my addr 172.21.15.22:0/4214446786 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f133e59c640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1340072030 0x7f134015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f133ed9d640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1340145a00 0x7f134013e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 -- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1340072030 msgr2=0x7f134015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1340072030 0x7f134015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 -- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1340145a00 msgr2=0x7f134013e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1340145a00 0x7f134013e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 -- 172.21.15.22:0/4214446786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13400b2510 con 0x7f13401454b0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1335d9b640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1328019090 tx=0x7f1328019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280ba070 con 0x7f13401454b0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280b5460 con 0x7f13401454b0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13400b27f0 con 0x7f13401454b0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b0040 con 0x7f13401454b0 2024-04-27T07:54:05.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13400b1650 con 0x7f13401454b0 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13400b1940 con 0x7f13401454b0 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f13280bd070 con 0x7f13401454b0 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 0x7f132006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.865+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1328120bd0 con 0x7f13401454b0 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.869+0000 7f133e59c640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 0x7f132006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:05.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.869+0000 7f133e59c640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 0x7f132006a840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f134006f100 tx=0x7f1330008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:05.873+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13400b1940 con 0x7f13401454b0 2024-04-27T07:54:06.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1340068c30 con 0x7f1320068380 2024-04-27T07:54:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f13377fe640 1 -- 172.21.15.22:0/4214446786 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1340068c30 con 0x7f1320068380 2024-04-27T07:54:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 3 to deep-scrub 2024-04-27T07:54:06.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 msgr2=0x7f132006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 0x7f132006a840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f134006f100 tx=0x7f1330008040 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 msgr2=0x7f13400b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1328019090 tx=0x7f1328019220 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 shutdown_connections 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1320068380 0x7f132006a840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1340145a00 0x7f134013e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13401454b0 0x7f13400b1b20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 --2- 172.21.15.22:0/4214446786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1340072030 0x7f134015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 >> 172.21.15.22:0/4214446786 conn(0x7f1340067f10 msgr2=0x7f134013fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 shutdown_connections 2024-04-27T07:54:06.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.045+0000 7f1344eac640 1 -- 172.21.15.22:0/4214446786 wait complete. 2024-04-27T07:54:06.067 INFO:tasks.ceph:Scrubbing osd.4 2024-04-27T07:54:06.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:06.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 Processor -- start 2024-04-27T07:54:06.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 -- start start 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09ae10 0x7f1ccc09b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ccc0cea80 con 0x7f1ccc0d0510 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1ccc0ced60 con 0x7f1ccc09aa30 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd2d02640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc0b3700 con 0x7f1ccc09ae10 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd0a9e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd0a9e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57820/0 (socket says 172.21.15.22:57820) 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.193+0000 7f1cd0a9e640 1 -- 172.21.15.22:0/2122232518 learned_addr learned my addr 172.21.15.22:0/2122232518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1ccbfff640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09ae10 0x7f1ccc09b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd0a9e640 1 -- 172.21.15.22:0/2122232518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09ae10 msgr2=0x7f1ccc09b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd0a9e640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09ae10 0x7f1ccc09b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd0a9e640 1 -- 172.21.15.22:0/2122232518 >> v1:172.21.15.183:6789/0 conn(0x7f1ccc0d0510 legacy=0x7f1ccc0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd0a9e640 1 -- 172.21.15.22:0/2122232518 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc0cdeb0 con 0x7f1ccc09aa30 2024-04-27T07:54:06.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd0a9e640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009870 tx=0x7f1cbc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fdd23c47f476b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1ccb7fe640 1 -- 172.21.15.22:0/2122232518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0b0070 con 0x7f1ccc09aa30 2024-04-27T07:54:06.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1ccb7fe640 1 -- 172.21.15.22:0/2122232518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cbc004230 con 0x7f1ccc09aa30 2024-04-27T07:54:06.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1ccb7fe640 1 -- 172.21.15.22:0/2122232518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc003620 con 0x7f1ccc09aa30 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2122232518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09aa30 msgr2=0x7f1ccc0b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009870 tx=0x7f1cbc01c440 comp rx=0 tx=0).stop 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2122232518 shutdown_connections 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09ae10 0x7f1ccc09b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2122232518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09aa30 0x7f1ccc0b39e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2122232518 >> 172.21.15.22:0/2122232518 conn(0x7f1ccc0b4100 msgr2=0x7f1ccc0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2122232518 shutdown_connections 2024-04-27T07:54:06.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.197+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2122232518 wait complete. 2024-04-27T07:54:06.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 Processor -- start 2024-04-27T07:54:06.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- start start 2024-04-27T07:54:06.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09aa30 0x7f1ccc0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 0x7f1ccc150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ccc0d0510 0x7f1ccc0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc0ce550 con 0x7f1ccc09ae10 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc0ce3d0 con 0x7f1ccc0d0510 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc169df0 con 0x7f1ccc09aa30 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 0x7f1ccc150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 0x7f1ccc150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57826/0 (socket says 172.21.15.22:57826) 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 -- 172.21.15.22:0/2152926931 learned_addr learned my addr 172.21.15.22:0/2152926931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd129f640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ccc0d0510 0x7f1ccc0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd0a9e640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09aa30 0x7f1ccc0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 -- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09aa30 msgr2=0x7f1ccc0b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ccc09aa30 0x7f1ccc0b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 -- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ccc0d0510 msgr2=0x7f1ccc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ccc0d0510 0x7f1ccc0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc0b78b0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1ccbfff640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 0x7f1ccc150d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000d5f0 tx=0x7f1cc000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00125f0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc0019070 con 0x7f1ccc09ae10 2024-04-27T07:54:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc0b7b90 con 0x7f1ccc09ae10 2024-04-27T07:54:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00129b0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc0b8150 con 0x7f1ccc09ae10 2024-04-27T07:54:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ccc1585c0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.201+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1cc0012b50 con 0x7f1ccc09ae10 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1cb8068380 0x7f1cb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1cc0086ad0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7f1cb806e240 0x7f1cb8070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cb8070d10 con 0x7f1cb806e240 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1cc008b2c0 con 0x7f1ccc09ae10 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cd0a9e640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1cb8068380 0x7f1cb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cd129f640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7f1cb806e240 0x7f1cb8070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cd129f640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7f1cb806e240 0x7f1cb8070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cd0a9e640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1cb8068380 0x7f1cb806a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ad000 tx=0x7f1cbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.205+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== osd.4 v2:172.21.15.183:6800/1968327270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cb8070d10 con 0x7f1cb806e240 2024-04-27T07:54:06.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 --> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1ccc04ff10 con 0x7f1cb806e240 2024-04-27T07:54:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cc97fa640 1 -- 172.21.15.22:0/2152926931 <== osd.4 v2:172.21.15.183:6800/1968327270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1ccc04ff10 con 0x7f1cb806e240 2024-04-27T07:54:06.224 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:06.224 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7f1cb806e240 msgr2=0x7f1cb8070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6800/1968327270,v1:172.21.15.183:6801/1968327270] conn(0x7f1cb806e240 0x7f1cb8070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1cb8068380 msgr2=0x7f1cb806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1cb8068380 0x7f1cb806a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ad000 tx=0x7f1cbc008040 comp rx=0 tx=0).stop 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 msgr2=0x7f1ccc150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 --2- 172.21.15.22:0/2152926931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ccc09ae10 0x7f1ccc150d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000d5f0 tx=0x7f1cc000dab0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd129f640 1 -- 172.21.15.22:0/2152926931 reap_dead start 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 shutdown_connections 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 >> 172.21.15.22:0/2152926931 conn(0x7f1ccc0b4100 msgr2=0x7f1ccc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 shutdown_connections 2024-04-27T07:54:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.221+0000 7f1cd2d02640 1 -- 172.21.15.22:0/2152926931 wait complete. 2024-04-27T07:54:06.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-27T07:54:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 Processor -- start 2024-04-27T07:54:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 -- start start 2024-04-27T07:54:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a0c0cfe10 0x7f1a0c1660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c0720d0 con 0x7f1a0c144dd0 2024-04-27T07:54:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c072370 con 0x7f1a0c06f870 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a142fc640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c0d0210 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12098640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1a0c144dd0 0x7f1a0c06f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50666/0 (socket says 172.21.15.22:50666) 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12098640 1 -- 172.21.15.22:0/924829272 learned_addr learned my addr 172.21.15.22:0/924829272 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12899640 1 --2- 172.21.15.22:0/924829272 >> v2:172.21.15.22:3300/0 conn(0x7f1a0c0cfe10 0x7f1a0c1660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a11096640 1 -- 172.21.15.22:0/924829272 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353159845 0 0) 0x7f1a0c0720d0 con 0x7f1a0c144dd0 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a11096640 1 -- 172.21.15.22:0/924829272 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19f8003530 con 0x7f1a0c144dd0 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12899640 1 -- 172.21.15.22:0/924829272 >> v1:172.21.15.183:6789/0 conn(0x7f1a0c06f870 legacy=0x7f1a0c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12899640 1 -- 172.21.15.22:0/924829272 >> v1:172.21.15.22:6789/0 conn(0x7f1a0c144dd0 legacy=0x7f1a0c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.349+0000 7f1a12899640 1 -- 172.21.15.22:0/924829272 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c072650 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a12899640 1 --2- 172.21.15.22:0/924829272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c0cfe10 0x7f1a0c1660e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009d90 tx=0x7f1a0401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d2b3ba0ae1d37b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a11096640 1 -- 172.21.15.22:0/924829272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a040b2070 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a11096640 1 -- 172.21.15.22:0/924829272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a040a5920 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a11096640 1 -- 172.21.15.22:0/924829272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a5cf0 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- 172.21.15.22:0/924829272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c0cfe10 msgr2=0x7f1a0c1660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 --2- 172.21.15.22:0/924829272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c0cfe10 0x7f1a0c1660e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009d90 tx=0x7f1a0401c970 comp rx=0 tx=0).stop 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- 172.21.15.22:0/924829272 shutdown_connections 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 --2- 172.21.15.22:0/924829272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c0cfe10 0x7f1a0c1660e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- 172.21.15.22:0/924829272 >> 172.21.15.22:0/924829272 conn(0x7f1a0c067f10 msgr2=0x7f1a0c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- 172.21.15.22:0/924829272 shutdown_connections 2024-04-27T07:54:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- 172.21.15.22:0/924829272 wait complete. 2024-04-27T07:54:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 Processor -- start 2024-04-27T07:54:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- start start 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c0cfe10 0x7f1a0c142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c144dd0 0x7f1a0c13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b4f60 con 0x7f1a0c06f870 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b4de0 con 0x7f1a0c144dd0 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a142fc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c168e40 con 0x7f1a0c0cfe10 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a12098640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.353+0000 7f1a12098640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57850/0 (socket says 172.21.15.22:57850) 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 -- 172.21.15.22:0/1001765712 learned_addr learned my addr 172.21.15.22:0/1001765712 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a11897640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c0cfe10 0x7f1a0c142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12899640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c144dd0 0x7f1a0c13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 -- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c0cfe10 msgr2=0x7f1a0c142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c0cfe10 0x7f1a0c142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 -- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c144dd0 msgr2=0x7f1a0c13fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c144dd0 0x7f1a0c13fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 -- 172.21.15.22:0/1001765712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c145610 con 0x7f1a0c06f870 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a12098640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c450 tx=0x7f1a0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08017070 con 0x7f1a0c06f870 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a08005cf0 con 0x7f1a0c06f870 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c1458f0 con 0x7f1a0c06f870 2024-04-27T07:54:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08007270 con 0x7f1a0c06f870 2024-04-27T07:54:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c145eb0 con 0x7f1a0c06f870 2024-04-27T07:54:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c068c30 con 0x7f1a0c06f870 2024-04-27T07:54:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1a0801a070 con 0x7f1a0c06f870 2024-04-27T07:54:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 0x7f19f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1a08085890 con 0x7f1a0c06f870 2024-04-27T07:54:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a11897640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 0x7f19f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.357+0000 7f1a11897640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 0x7f19f006a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c1473b0 tx=0x7f19fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.361+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a08055b70 con 0x7f1a0c06f870 2024-04-27T07:54:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.521+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1a0c1453c0 con 0x7f19f0068380 2024-04-27T07:54:06.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a02ffd640 1 -- 172.21.15.22:0/1001765712 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1a0c1453c0 con 0x7f19f0068380 2024-04-27T07:54:06.527 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 4 to deep-scrub 2024-04-27T07:54:06.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 msgr2=0x7f19f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 0x7f19f006a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c1473b0 tx=0x7f19fc009040 comp rx=0 tx=0).stop 2024-04-27T07:54:06.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 msgr2=0x7f1a0c1398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c450 tx=0x7f1a0800c910 comp rx=0 tx=0).stop 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 shutdown_connections 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f19f0068380 0x7f19f006a840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c144dd0 0x7f1a0c13fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c0cfe10 0x7f1a0c142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 --2- 172.21.15.22:0/1001765712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0c06f870 0x7f1a0c1398f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 >> 172.21.15.22:0/1001765712 conn(0x7f1a0c067f10 msgr2=0x7f1a0c0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 shutdown_connections 2024-04-27T07:54:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.525+0000 7f1a142fc640 1 -- 172.21.15.22:0/1001765712 wait complete. 2024-04-27T07:54:06.546 INFO:tasks.ceph:Scrubbing osd.5 2024-04-27T07:54:06.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:06.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 Processor -- start 2024-04-27T07:54:06.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 -- start start 2024-04-27T07:54:06.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6cb00d1940 0x7f6cb00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 0x7f6cb00d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb01635a0 con 0x7f6cb0161fa0 2024-04-27T07:54:06.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb00cfe00 con 0x7f6cb00d1940 2024-04-27T07:54:06.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.669+0000 7f6cb6639640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00d5680 con 0x7f6cb0163130 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caffff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6cb0161fa0 0x7f6cb00d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.22:42764/0 (socket says 172.21.15.22:42764) 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caffff640 1 -- 172.21.15.22:0/3276289882 learned_addr learned my addr 172.21.15.22:0/3276289882 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3276289882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 0x7f6cb00d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caf7fe640 1 --2- 172.21.15.22:0/3276289882 >> v2:172.21.15.183:3300/0 conn(0x7f6cb00d1940 0x7f6cb00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caeffd640 1 -- 172.21.15.22:0/3276289882 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444584418 0 0) 0x7f6ca0001120 con 0x7f6cb0161fa0 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caeffd640 1 -- 172.21.15.22:0/3276289882 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c98003530 con 0x7f6cb0161fa0 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 -- 172.21.15.22:0/3276289882 >> v2:172.21.15.183:3300/0 conn(0x7f6cb00d1940 msgr2=0x7f6cb00d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3276289882 >> v2:172.21.15.183:3300/0 conn(0x7f6cb00d1940 0x7f6cb00d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 -- 172.21.15.22:0/3276289882 >> v1:172.21.15.183:6789/0 conn(0x7f6cb0161fa0 legacy=0x7f6cb00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 -- 172.21.15.22:0/3276289882 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00cf4d0 con 0x7f6cb0163130 2024-04-27T07:54:06.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3276289882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 0x7f6cb00d4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009d90 tx=0x7f6ca801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184ddb26dda6ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caeffd640 1 -- 172.21.15.22:0/3276289882 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80b2070 con 0x7f6cb0163130 2024-04-27T07:54:06.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caeffd640 1 -- 172.21.15.22:0/3276289882 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca80a5890 con 0x7f6cb0163130 2024-04-27T07:54:06.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.673+0000 7f6caeffd640 1 -- 172.21.15.22:0/3276289882 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80a5c60 con 0x7f6cb0163130 2024-04-27T07:54:06.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- 172.21.15.22:0/3276289882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 msgr2=0x7f6cb00d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3276289882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 0x7f6cb00d4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009d90 tx=0x7f6ca801c8e0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- 172.21.15.22:0/3276289882 shutdown_connections 2024-04-27T07:54:06.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3276289882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb0163130 0x7f6cb00d4f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3276289882 >> v2:172.21.15.183:3300/0 conn(0x7f6cb00d1940 0x7f6cb00d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- 172.21.15.22:0/3276289882 >> 172.21.15.22:0/3276289882 conn(0x7f6cb008bfc0 msgr2=0x7f6cb00d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- 172.21.15.22:0/3276289882 shutdown_connections 2024-04-27T07:54:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- 172.21.15.22:0/3276289882 wait complete. 2024-04-27T07:54:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 Processor -- start 2024-04-27T07:54:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- start start 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb00d1940 0x7f6cb00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 0x7f6cb015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cb0163130 0x7f6cb00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00b8000 con 0x7f6cb0161fa0 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00b7e80 con 0x7f6cb0163130 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb6639640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00b8180 con 0x7f6cb00d1940 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 0x7f6cb015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 0x7f6cb015f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57862/0 (socket says 172.21.15.22:57862) 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 -- 172.21.15.22:0/3395770484 learned_addr learned my addr 172.21.15.22:0/3395770484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caffff640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb00d1940 0x7f6cb00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cb0163130 0x7f6cb00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 -- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb00d1940 msgr2=0x7f6cb00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb00d1940 0x7f6cb00caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 -- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cb0163130 msgr2=0x7f6cb00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cb0163130 0x7f6cb00cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00cb990 con 0x7f6cb0161fa0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cb0163130 0x7f6cb00cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caffff640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cb00d1940 0x7f6cb00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.677+0000 7f6caf7fe640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 0x7f6cb015f1a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e930 tx=0x7f6ca400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4017070 con 0x7f6cb0161fa0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca400ca00 con 0x7f6cb0161fa0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb00cd9f0 con 0x7f6cb0161fa0 2024-04-27T07:54:06.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400cdd0 con 0x7f6cb0161fa0 2024-04-27T07:54:06.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb00cbc30 con 0x7f6cb0161fa0 2024-04-27T07:54:06.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6cb004ff10 con 0x7f6cb0161fa0 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6ca401a070 con 0x7f6cb0161fa0 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6c9c068380 0x7f6c9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6ca4086aa0 con 0x7f6cb0161fa0 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f6c9c06e240 0x7f6c9c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c9c070d10 con 0x7f6c9c06e240 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6ca408b2c0 con 0x7f6cb0161fa0 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6caffff640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6c9c068380 0x7f6c9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f6c9c06e240 0x7f6c9c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6caffff640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6c9c068380 0x7f6c9c06a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0005890 tx=0x7f6ca0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.681+0000 7f6cb4bd6640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f6c9c06e240 0x7f6c9c070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.685+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== osd.5 v2:172.21.15.183:6816/2484095365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c9c070d10 con 0x7f6c9c06e240 2024-04-27T07:54:06.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 --> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6cb00d1010 con 0x7f6c9c06e240 2024-04-27T07:54:06.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cacff9640 1 -- 172.21.15.22:0/3395770484 <== osd.5 v2:172.21.15.183:6816/2484095365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6cb00d1010 con 0x7f6c9c06e240 2024-04-27T07:54:06.702 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:06.702 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f6c9c06e240 msgr2=0x7f6c9c070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6816/2484095365,v1:172.21.15.183:6817/2484095365] conn(0x7f6c9c06e240 0x7f6c9c070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6c9c068380 msgr2=0x7f6c9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f6c9c068380 0x7f6c9c06a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0005890 tx=0x7f6ca0009040 comp rx=0 tx=0).stop 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 msgr2=0x7f6cb015f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 --2- 172.21.15.22:0/3395770484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb0161fa0 0x7f6cb015f1a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e930 tx=0x7f6ca400edf0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb4bd6640 1 -- 172.21.15.22:0/3395770484 reap_dead start 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 shutdown_connections 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 >> 172.21.15.22:0/3395770484 conn(0x7f6cb008bfc0 msgr2=0x7f6cb00b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 shutdown_connections 2024-04-27T07:54:06.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.697+0000 7f6cb6639640 1 -- 172.21.15.22:0/3395770484 wait complete. 2024-04-27T07:54:06.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-27T07:54:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 Processor -- start 2024-04-27T07:54:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 -- start start 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1bb80735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bb806c9f0 con 0x7f1bb80d1700 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bb80ae740 con 0x7f1bb80ce930 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bc0074640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb80cfbd0 con 0x7f1bb80d1ae0 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbde10640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1bb80d1700 0x7f1bb806bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50680/0 (socket says 172.21.15.22:50680) 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbde10640 1 -- 172.21.15.22:0/2550730853 learned_addr learned my addr 172.21.15.22:0/2550730853 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbd60f640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1bb80735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018975499 0 0) 0x7f1bb806c9f0 con 0x7f1bb80d1700 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ba0003530 con 0x7f1bb80d1700 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3325625993 0 0) 0x7f1ba0003530 con 0x7f1bb80d1700 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 msgr2=0x7f1bb80735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1bb80735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 >> v1:172.21.15.183:6789/0 conn(0x7f1bb80ce930 legacy=0x7f1bb806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb80cefc0 con 0x7f1bb80d1700 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f1ba8001680 con 0x7f1bb80d1700 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 >> v1:172.21.15.22:6789/0 conn(0x7f1bb80d1700 legacy=0x7f1bb806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba00039f0 0x7f1ba0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1ba0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1ba00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb80cfbd0 con 0x7f1ba0004d50 2024-04-27T07:54:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba000bb30 con 0x7f1ba00039f0 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba000bd30 con 0x7f1bb80d1ae0 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1ba00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbde10640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba00039f0 0x7f1ba0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbd60f640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1ba0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 -- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 msgr2=0x7f1ba0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb80d1ae0 0x7f1ba0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 -- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba00039f0 msgr2=0x7f1ba0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba00039f0 0x7f1ba0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.825+0000 7f1bbe611640 1 -- 172.21.15.22:0/2550730853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0003530 con 0x7f1ba0004d50 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bbe611640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1ba00051d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0001780 tx=0x7f1bb001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af8504dfcf61f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00013a0 con 0x7f1ba0004d50 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00052f0 con 0x7f1ba0004d50 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bbce0e640 1 -- 172.21.15.22:0/2550730853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003840 con 0x7f1ba0004d50 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- 172.21.15.22:0/2550730853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 msgr2=0x7f1ba00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 --2- 172.21.15.22:0/2550730853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1ba00051d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0001780 tx=0x7f1bb001c420 comp rx=0 tx=0).stop 2024-04-27T07:54:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bbe611640 1 -- 172.21.15.22:0/2550730853 reap_dead start 2024-04-27T07:54:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- 172.21.15.22:0/2550730853 shutdown_connections 2024-04-27T07:54:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- 172.21.15.22:0/2550730853 >> 172.21.15.22:0/2550730853 conn(0x7f1bb80b6b60 msgr2=0x7f1bb809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- 172.21.15.22:0/2550730853 shutdown_connections 2024-04-27T07:54:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- 172.21.15.22:0/2550730853 wait complete. 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 Processor -- start 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 -- start start 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.829+0000 7f1bc0074640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb815e160 0x7f1bb80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 0x7f1bb81f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb80cf860 con 0x7f1ba0004d50 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb80cf580 con 0x7f1bb815e160 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb80afca0 con 0x7f1bb81f57c0 2024-04-27T07:54:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57866/0 (socket says 172.21.15.22:57866) 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 -- 172.21.15.22:0/3751784870 learned_addr learned my addr 172.21.15.22:0/3751784870 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbd60f640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb815e160 0x7f1bb80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbe611640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 0x7f1bb81f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 -- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 msgr2=0x7f1bb81f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 0x7f1bb81f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 -- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb815e160 msgr2=0x7f1bb80b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb815e160 0x7f1bb80b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 -- 172.21.15.22:0/3751784870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb8151390 con 0x7f1ba0004d50 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbe611640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 0x7f1bb81f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbde10640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8011900 tx=0x7f1ba8011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8010040 con 0x7f1ba0004d50 2024-04-27T07:54:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba801a070 con 0x7f1ba0004d50 2024-04-27T07:54:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb8158280 con 0x7f1ba0004d50 2024-04-27T07:54:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800fdc0 con 0x7f1ba0004d50 2024-04-27T07:54:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81559b0 con 0x7f1ba0004d50 2024-04-27T07:54:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb8068c30 con 0x7f1ba0004d50 2024-04-27T07:54:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1ba801f460 con 0x7f1ba0004d50 2024-04-27T07:54:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 0x7f1ba406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1ba8089980 con 0x7f1ba0004d50 2024-04-27T07:54:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbd60f640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 0x7f1ba406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.833+0000 7f1bbd60f640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 0x7f1ba406a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80b3210 tx=0x7f1bac00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:06.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.837+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ba805a240 con 0x7f1ba0004d50 2024-04-27T07:54:06.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.993+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f1bb8070450 con 0x7f1ba4068380 2024-04-27T07:54:06.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bb67fc640 1 -- 172.21.15.22:0/3751784870 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1bb8070450 con 0x7f1ba4068380 2024-04-27T07:54:06.999 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 5 to deep-scrub 2024-04-27T07:54:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 msgr2=0x7f1ba406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 0x7f1ba406a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80b3210 tx=0x7f1bac00b040 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 msgr2=0x7f1bb8150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8011900 tx=0x7f1ba8011dc0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 shutdown_connections 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f1ba4068380 0x7f1ba406a840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb81f57c0 0x7f1bb81f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb815e160 0x7f1bb80b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 --2- 172.21.15.22:0/3751784870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba0004d50 0x7f1bb8150e50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 >> 172.21.15.22:0/3751784870 conn(0x7f1bb80b6b60 msgr2=0x7f1bb815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 shutdown_connections 2024-04-27T07:54:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:06.997+0000 7f1bc0074640 1 -- 172.21.15.22:0/3751784870 wait complete. 2024-04-27T07:54:07.018 INFO:tasks.ceph:Scrubbing osd.6 2024-04-27T07:54:07.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 Processor -- start 2024-04-27T07:54:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 -- start start 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 0x7ff104167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1040cdb10 con 0x7ff104071f70 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1040cddf0 con 0x7ff1040cffe0 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff10b248640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff1040ce0d0 con 0x7ff10406f750 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff108fe4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff104071f70 0x7ff104072350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50692/0 (socket says 172.21.15.22:50692) 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff108fe4640 1 -- 172.21.15.22:0/154423415 learned_addr learned my addr 172.21.15.22:0/154423415 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 0x7ff104167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896914887 0 0) 0x7ff1040cdb10 con 0x7ff104071f70 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0f0003530 con 0x7ff104071f70 2024-04-27T07:54:07.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2634370708 0 0) 0x7ff0f0003530 con 0x7ff104071f70 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 msgr2=0x7ff104167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 0x7ff104167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 >> v1:172.21.15.183:6789/0 conn(0x7ff1040cffe0 legacy=0x7ff104164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10406e220 con 0x7ff104071f70 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7ff0fc001a00 con 0x7ff104071f70 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 >> v1:172.21.15.22:6789/0 conn(0x7ff104071f70 legacy=0x7ff104072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f00039f0 0x7ff0f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 0x7ff0f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0004d50 0x7ff0f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040ce0d0 con 0x7ff0f0004d50 2024-04-27T07:54:07.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f000bb00 con 0x7ff0f00039f0 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f000bd00 con 0x7ff10406f750 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0004d50 0x7ff0f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff108fe4640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f00039f0 0x7ff0f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 -- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 msgr2=0x7ff0f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10406f750 0x7ff0f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 -- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f00039f0 msgr2=0x7ff0f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f00039f0 0x7ff0f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.145+0000 7ff1097e5640 1 -- 172.21.15.22:0/154423415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f0003530 con 0x7ff0f0004d50 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff1097e5640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0004d50 0x7ff0f00051d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009ab0 tx=0x7ff0f401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1390b314f3140 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40b2070 con 0x7ff0f0004d50 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f40a5570 con 0x7ff0f0004d50 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff1037fe640 1 -- 172.21.15.22:0/154423415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40096b0 con 0x7ff0f0004d50 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 -- 172.21.15.22:0/154423415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0004d50 msgr2=0x7ff0f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 --2- 172.21.15.22:0/154423415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0004d50 0x7ff0f00051d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009ab0 tx=0x7ff0f401ca90 comp rx=0 tx=0).stop 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff1097e5640 1 -- 172.21.15.22:0/154423415 reap_dead start 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 -- 172.21.15.22:0/154423415 shutdown_connections 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 -- 172.21.15.22:0/154423415 >> 172.21.15.22:0/154423415 conn(0x7ff1040b21d0 msgr2=0x7ff1040b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 -- 172.21.15.22:0/154423415 shutdown_connections 2024-04-27T07:54:07.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.149+0000 7ff10b248640 1 -- 172.21.15.22:0/154423415 wait complete. 2024-04-27T07:54:07.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 Processor -- start 2024-04-27T07:54:07.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- start start 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0004d50 0x7ff10415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 0x7ff104141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10413ab80 0x7ff10413b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff10406dfd0 con 0x7ff104140be0 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040cd8c0 con 0x7ff0f0004d50 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff104072fb0 con 0x7ff10413ab80 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 0x7ff104141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 0x7ff104141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57886/0 (socket says 172.21.15.22:57886) 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 -- 172.21.15.22:0/3103295179 learned_addr learned my addr 172.21.15.22:0/3103295179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:07.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 -- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10413ab80 msgr2=0x7ff10413b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff10413ab80 0x7ff10413b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 -- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0004d50 msgr2=0x7ff10415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0004d50 0x7ff10415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10413b8b0 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff103fff640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 0x7ff104141060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c530 tx=0x7ff0ec00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec017070 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0052d0 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10413bb90 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec003840 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1041f2ee0 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff10404ff10 con 0x7ff104140be0 2024-04-27T07:54:07.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff0ec01a070 con 0x7ff104140be0 2024-04-27T07:54:07.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.153+0000 7ff1017fa640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff0e4068380 0x7ff0e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7ff0ec0862a0 con 0x7ff104140be0 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1017fa640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7ff0e406e240 0x7ff0e4070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0e4070d10 con 0x7ff0e406e240 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff0ec086650 con 0x7ff104140be0 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff108fe4640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff0e4068380 0x7ff0e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1097e5640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7ff0e406e240 0x7ff0e4070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff108fe4640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff0e4068380 0x7ff0e406a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc004c00 tx=0x7ff0fc004b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1097e5640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7ff0e406e240 0x7ff0e4070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.157+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== osd.6 v2:172.21.15.183:6808/3842666624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0e4070d10 con 0x7ff0e406e240 2024-04-27T07:54:07.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.169+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 --> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff1040af9c0 con 0x7ff0e406e240 2024-04-27T07:54:07.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff1017fa640 1 -- 172.21.15.22:0/3103295179 <== osd.6 v2:172.21.15.183:6808/3842666624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff1040af9c0 con 0x7ff0e406e240 2024-04-27T07:54:07.176 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:07.176 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7ff0e406e240 msgr2=0x7ff0e4070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6808/3842666624,v1:172.21.15.183:6809/3842666624] conn(0x7ff0e406e240 0x7ff0e4070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff0e4068380 msgr2=0x7ff0e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7ff0e4068380 0x7ff0e406a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc004c00 tx=0x7ff0fc004b70 comp rx=0 tx=0).stop 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 msgr2=0x7ff104141060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 --2- 172.21.15.22:0/3103295179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff104140be0 0x7ff104141060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c530 tx=0x7ff0ec00c9f0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff1097e5640 1 -- 172.21.15.22:0/3103295179 reap_dead start 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 shutdown_connections 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 >> 172.21.15.22:0/3103295179 conn(0x7ff1040b21d0 msgr2=0x7ff104143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 shutdown_connections 2024-04-27T07:54:07.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.173+0000 7ff10b248640 1 -- 172.21.15.22:0/3103295179 wait complete. 2024-04-27T07:54:07.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-27T07:54:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 Processor -- start 2024-04-27T07:54:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 -- start start 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f416c1660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c0720d0 con 0x7f416c06f870 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c072370 con 0x7f416c144dd0 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4170ae6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f416c072650 con 0x7f416c0cfe10 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4161d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f416c06f870 0x7f416c0cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50706/0 (socket says 172.21.15.22:50706) 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4161d9b640 1 -- 172.21.15.22:0/3919709756 learned_addr learned my addr 172.21.15.22:0/3919709756 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f416c1660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121528903 0 0) 0x7f416c0720d0 con 0x7f416c06f870 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4158003530 con 0x7f416c06f870 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526999602 0 0) 0x7f4158003530 con 0x7f416c06f870 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 msgr2=0x7f416c1660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f416c1660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 >> v1:172.21.15.183:6789/0 conn(0x7f416c144dd0 legacy=0x7f416c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c072930 con 0x7f416c06f870 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7f41540016a0 con 0x7f416c06f870 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 >> v1:172.21.15.22:6789/0 conn(0x7f416c06f870 legacy=0x7f416c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41580039f0 0x7f4158003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f4158004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4158004d50 0x7f41580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f416c072650 con 0x7f4158004d50 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f415800ba80 con 0x7f41580039f0 2024-04-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f415800bc80 con 0x7f416c0cfe10 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4158004d50 0x7f41580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f4161d9b640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f4158004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416a59c640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41580039f0 0x7f4158003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 -- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 msgr2=0x7f4158004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f416c0cfe10 0x7f4158004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 -- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41580039f0 msgr2=0x7f4158003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41580039f0 0x7f4158003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416ad9d640 1 -- 172.21.15.22:0/3919709756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4158003530 con 0x7f4158004d50 2024-04-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.297+0000 7f416a59c640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41580039f0 0x7f4158003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4158004d50 0x7f41580051d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f90 tx=0x7f416401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8f0a03717c7760 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640b2070 con 0x7f4158004d50 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41640a5c50 con 0x7f4158004d50 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4169d9b640 1 -- 172.21.15.22:0/3919709756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164009c10 con 0x7f4158004d50 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 -- 172.21.15.22:0/3919709756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4158004d50 msgr2=0x7f41580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3919709756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4158004d50 0x7f41580051d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f90 tx=0x7f416401cc40 comp rx=0 tx=0).stop 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f416ad9d640 1 -- 172.21.15.22:0/3919709756 reap_dead start 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 -- 172.21.15.22:0/3919709756 shutdown_connections 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 -- 172.21.15.22:0/3919709756 >> 172.21.15.22:0/3919709756 conn(0x7f416c067f10 msgr2=0x7f416c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 -- 172.21.15.22:0/3919709756 shutdown_connections 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.301+0000 7f4170ae6640 1 -- 172.21.15.22:0/3919709756 wait complete. 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 Processor -- start 2024-04-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- start start 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4158004d50 0x7f416c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 0x7f416c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0b4de0 con 0x7f416c13d5c0 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1451b0 con 0x7f416c15ad90 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0b4f60 con 0x7f4158004d50 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57912/0 (socket says 172.21.15.22:57912) 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 -- 172.21.15.22:0/3922548781 learned_addr learned my addr 172.21.15.22:0/3922548781 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416a59c640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4158004d50 0x7f416c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4161d9b640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 0x7f416c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 -- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4158004d50 msgr2=0x7f416c0abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4158004d50 0x7f416c0abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 -- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 msgr2=0x7f416c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 0x7f416c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 -- 172.21.15.22:0/3922548781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c0a9490 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4161d9b640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 0x7f416c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f416ad9d640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f90 tx=0x7f4164009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640bd070 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41640a5a50 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640b2040 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c13d200 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c13bc90 con 0x7f416c13d5c0 2024-04-27T07:54:07.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c068c30 con 0x7f416c13d5c0 2024-04-27T07:54:07.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.305+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f41640c0070 con 0x7f416c13d5c0 2024-04-27T07:54:07.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.309+0000 7f41637fe640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 0x7f414c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.309+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f41641205d0 con 0x7f416c13d5c0 2024-04-27T07:54:07.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.309+0000 7f416a59c640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 0x7f414c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.309+0000 7f416a59c640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 0x7f414c06a870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f415c000c10 tx=0x7f415c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.313+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f41640c7020 con 0x7f416c13d5c0 2024-04-27T07:54:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f416c06be20 con 0x7f414c0683b0 2024-04-27T07:54:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f41637fe640 1 -- 172.21.15.22:0/3922548781 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f416c06be20 con 0x7f414c0683b0 2024-04-27T07:54:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 6 to deep-scrub 2024-04-27T07:54:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 msgr2=0x7f414c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 0x7f414c06a870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f415c000c10 tx=0x7f415c009040 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 msgr2=0x7f416c142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f90 tx=0x7f4164009750 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 shutdown_connections 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f414c0683b0 0x7f414c06a870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c13d5c0 0x7f416c142f50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f416c15ad90 0x7f416c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 --2- 172.21.15.22:0/3922548781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4158004d50 0x7f416c0abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.473+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 >> 172.21.15.22:0/3922548781 conn(0x7f416c067f10 msgr2=0x7f416c15d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.477+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 shutdown_connections 2024-04-27T07:54:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.477+0000 7f4170ae6640 1 -- 172.21.15.22:0/3922548781 wait complete. 2024-04-27T07:54:07.496 INFO:tasks.ceph:Scrubbing osd.7 2024-04-27T07:54:07.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 Processor -- start 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 -- start start 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f11e406f9f0 0x7f11e40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f11e40cd820 0x7f11e40cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e4163380 con 0x7f11e4071f70 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f11e40cd390 con 0x7f11e40cd820 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e9a0b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f11e40b3bd0 con 0x7f11e406f9f0 2024-04-27T07:54:07.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f11e40cd820 0x7f11e40cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f11e40cd820 0x7f11e40cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57926/0 (socket says 172.21.15.22:57926) 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 -- 172.21.15.22:0/1521120963 learned_addr learned my addr 172.21.15.22:0/1521120963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e27fc640 1 --2- 172.21.15.22:0/1521120963 >> v2:172.21.15.183:3300/0 conn(0x7f11e406f9f0 0x7f11e40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 -- 172.21.15.22:0/1521120963 >> v2:172.21.15.183:3300/0 conn(0x7f11e406f9f0 msgr2=0x7f11e40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 --2- 172.21.15.22:0/1521120963 >> v2:172.21.15.183:3300/0 conn(0x7f11e406f9f0 0x7f11e40b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 -- 172.21.15.22:0/1521120963 >> v1:172.21.15.183:6789/0 conn(0x7f11e4071f70 legacy=0x7f11e4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.625+0000 7f11e37fe640 1 -- 172.21.15.22:0/1521120963 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e40b4080 con 0x7f11e40cd820 2024-04-27T07:54:07.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e37fe640 1 --2- 172.21.15.22:0/1521120963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40cdcb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b5450 tx=0x7f11dc01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30ccff6b021f531 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e1ffb640 1 -- 172.21.15.22:0/1521120963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0b4070 con 0x7f11e40cd820 2024-04-27T07:54:07.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e1ffb640 1 -- 172.21.15.22:0/1521120963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11dc0af400 con 0x7f11e40cd820 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e1ffb640 1 -- 172.21.15.22:0/1521120963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0af7d0 con 0x7f11e40cd820 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- 172.21.15.22:0/1521120963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 msgr2=0x7f11e40cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/1521120963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40cdcb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b5450 tx=0x7f11dc01f1a0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- 172.21.15.22:0/1521120963 shutdown_connections 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/1521120963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40cdcb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/1521120963 >> v2:172.21.15.183:3300/0 conn(0x7f11e406f9f0 0x7f11e40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- 172.21.15.22:0/1521120963 >> 172.21.15.22:0/1521120963 conn(0x7f11e40b21d0 msgr2=0x7f11e40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- 172.21.15.22:0/1521120963 shutdown_connections 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- 172.21.15.22:0/1521120963 wait complete. 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 Processor -- start 2024-04-27T07:54:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- start start 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11e406f9f0 0x7f11e40c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4071f70 0x7f11e41f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11e40ce1f0 con 0x7f11e40cd820 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11e40b4640 con 0x7f11e4071f70 2024-04-27T07:54:07.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e9a0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f11e40ce370 con 0x7f11e406f9f0 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.629+0000 7f11e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57940/0 (socket says 172.21.15.22:57940) 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 -- 172.21.15.22:0/3842236931 learned_addr learned my addr 172.21.15.22:0/3842236931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e2ffd640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11e406f9f0 0x7f11e40c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e27fc640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4071f70 0x7f11e41f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 -- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11e406f9f0 msgr2=0x7f11e40c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11e406f9f0 0x7f11e40c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 -- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4071f70 msgr2=0x7f11e41f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4071f70 0x7f11e41f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e40aab70 con 0x7f11e40cd820 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e27fc640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4071f70 0x7f11e41f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e2ffd640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11e406f9f0 0x7f11e40c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e37fe640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40abc10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0b2040 tx=0x7f11dc01bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0b4040 con 0x7f11e40cd820 2024-04-27T07:54:07.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11dc0c1070 con 0x7f11e40cd820 2024-04-27T07:54:07.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc009510 con 0x7f11e40cd820 2024-04-27T07:54:07.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e40aae50 con 0x7f11e40cd820 2024-04-27T07:54:07.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e40ab410 con 0x7f11e40cd820 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11e404ff10 con 0x7f11e40cd820 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f11dc0b7070 con 0x7f11e40cd820 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f11d0068380 0x7f11d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f11dc125220 con 0x7f11e40cd820 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7f11d006e240 0x7f11d0070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11dc008ea0 con 0x7f11d006e240 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f11dc1256b0 con 0x7f11e40cd820 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e2ffd640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f11d0068380 0x7f11d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.633+0000 7f11e27fc640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7f11d006e240 0x7f11d0070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.637+0000 7f11e27fc640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7f11d006e240 0x7f11d0070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.637+0000 7f11e2ffd640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f11d0068380 0x7f11d006a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f11e41ef870 tx=0x7f11d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.637+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== osd.7 v2:172.21.15.183:6824/1208358952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11dc008ea0 con 0x7f11d006e240 2024-04-27T07:54:07.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.649+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 --> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f11e406f160 con 0x7f11d006e240 2024-04-27T07:54:07.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11cbfff640 1 -- 172.21.15.22:0/3842236931 <== osd.7 v2:172.21.15.183:6824/1208358952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f11e406f160 con 0x7f11d006e240 2024-04-27T07:54:07.655 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-27T07:54:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:54:07.655 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-27T07:54:07.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7f11d006e240 msgr2=0x7f11d0070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6824/1208358952,v1:172.21.15.183:6825/1208358952] conn(0x7f11d006e240 0x7f11d0070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f11d0068380 msgr2=0x7f11d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f11d0068380 0x7f11d006a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f11e41ef870 tx=0x7f11d4009040 comp rx=0 tx=0).stop 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 msgr2=0x7f11e40abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 --2- 172.21.15.22:0/3842236931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11e40cd820 0x7f11e40abc10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0b2040 tx=0x7f11dc01bf40 comp rx=0 tx=0).stop 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e37fe640 1 -- 172.21.15.22:0/3842236931 reap_dead start 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 shutdown_connections 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 >> 172.21.15.22:0/3842236931 conn(0x7f11e40b21d0 msgr2=0x7f11e40b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 shutdown_connections 2024-04-27T07:54:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.653+0000 7f11e9a0b640 1 -- 172.21.15.22:0/3842236931 wait complete. 2024-04-27T07:54:07.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-27T07:54:07.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 Processor -- start 2024-04-27T07:54:07.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 -- start start 2024-04-27T07:54:07.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 0x7fb0f406f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0f406f6a0 0x7fb0f40cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f40720d0 con 0x7fb0f40cfc40 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f406fb00 con 0x7fb0f406f6a0 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0fad52640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f4072370 con 0x7fb0f4144dd0 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0f3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0f406f6a0 0x7fb0f40cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0f92ef640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb0f40cfc40 0x7fb0f415baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50718/0 (socket says 172.21.15.22:50718) 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0f92ef640 1 -- 172.21.15.22:0/2371345930 learned_addr learned my addr 172.21.15.22:0/2371345930 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.781+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2371345930 >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 0x7fb0f406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f3fff640 1 -- 172.21.15.22:0/2371345930 >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 msgr2=0x7fb0f406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f3fff640 1 --2- 172.21.15.22:0/2371345930 >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 0x7fb0f406f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f3fff640 1 -- 172.21.15.22:0/2371345930 >> v1:172.21.15.22:6789/0 conn(0x7fb0f40cfc40 legacy=0x7fb0f415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f3fff640 1 -- 172.21.15.22:0/2371345930 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f4072650 con 0x7fb0f406f6a0 2024-04-27T07:54:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2371345930 >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 0x7fb0f406f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f3fff640 1 --2- 172.21.15.22:0/2371345930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f406f6a0 0x7fb0f40cf700 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b3da0 tx=0x7fb0e801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64929c4c185dc160 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f37fe640 1 -- 172.21.15.22:0/2371345930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80b0070 con 0x7fb0f406f6a0 2024-04-27T07:54:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f37fe640 1 -- 172.21.15.22:0/2371345930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e80ab600 con 0x7fb0f406f6a0 2024-04-27T07:54:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0f37fe640 1 -- 172.21.15.22:0/2371345930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80ab9d0 con 0x7fb0f406f6a0 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 -- 172.21.15.22:0/2371345930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f406f6a0 msgr2=0x7fb0f40cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2371345930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f406f6a0 0x7fb0f40cf700 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b3da0 tx=0x7fb0e801c480 comp rx=0 tx=0).stop 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 -- 172.21.15.22:0/2371345930 shutdown_connections 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2371345930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f406f6a0 0x7fb0f40cf700 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2371345930 >> v2:172.21.15.183:3300/0 conn(0x7fb0f4144dd0 0x7fb0f406f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 -- 172.21.15.22:0/2371345930 >> 172.21.15.22:0/2371345930 conn(0x7fb0f4067f10 msgr2=0x7fb0f406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 -- 172.21.15.22:0/2371345930 shutdown_connections 2024-04-27T07:54:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.785+0000 7fb0fad52640 1 -- 172.21.15.22:0/2371345930 wait complete. 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 Processor -- start 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- start start 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 0x7fb0f415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0f40cfc40 0x7fb0f41419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40d01d0 con 0x7fb0f4144dd0 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40d0050 con 0x7fb0f406f6a0 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f40b4de0 con 0x7fb0f40cfc40 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57956/0 (socket says 172.21.15.22:57956) 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 -- 172.21.15.22:0/2591400184 learned_addr learned my addr 172.21.15.22:0/2591400184 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 0x7fb0f415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 -- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0f40cfc40 msgr2=0x7fb0f41419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f3fff640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0f40cfc40 0x7fb0f41419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0f40cfc40 0x7fb0f41419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 -- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 msgr2=0x7fb0f415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 0x7fb0f415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 -- 172.21.15.22:0/2591400184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f4141500 con 0x7fb0f4144dd0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 0x7fb0f415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f92ef640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00cae0 tx=0x7fb0ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec017070 con 0x7fb0f4144dd0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec00ac40 con 0x7fb0f4144dd0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f41406b0 con 0x7fb0f4144dd0 2024-04-27T07:54:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec012400 con 0x7fb0f4144dd0 2024-04-27T07:54:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f4140c60 con 0x7fb0f4144dd0 2024-04-27T07:54:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f4068c30 con 0x7fb0f4144dd0 2024-04-27T07:54:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb0ec01a070 con 0x7fb0f4144dd0 2024-04-27T07:54:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 0x7fb0d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb0ec086450 con 0x7fb0f4144dd0 2024-04-27T07:54:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.789+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 0x7fb0d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.793+0000 7fb0f8aee640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 0x7fb0d806a840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0008890 tx=0x7fb0e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.793+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0ec056d10 con 0x7fb0f4144dd0 2024-04-27T07:54:07.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.957+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb0f40b5370 con 0x7fb0d8068380 2024-04-27T07:54:07.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.957+0000 7fb0f17fa640 1 -- 172.21.15.22:0/2591400184 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb0f40b5370 con 0x7fb0d8068380 2024-04-27T07:54:07.962 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 7 to deep-scrub 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 msgr2=0x7fb0d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 0x7fb0d806a840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0008890 tx=0x7fb0e0008040 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 msgr2=0x7fb0f4141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00cae0 tx=0x7fb0ec00a060 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 shutdown_connections 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb0d8068380 0x7fb0d806a840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f4144dd0 0x7fb0f4141ef0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0f40cfc40 0x7fb0f41419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 --2- 172.21.15.22:0/2591400184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0f406f6a0 0x7fb0f415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 >> 172.21.15.22:0/2591400184 conn(0x7fb0f4067f10 msgr2=0x7fb0f415baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 shutdown_connections 2024-04-27T07:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:07.961+0000 7fb0fad52640 1 -- 172.21.15.22:0/2591400184 wait complete. 2024-04-27T07:54:07.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:54:08.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 Processor -- start 2024-04-27T07:54:08.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 -- start start 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f405006f870 0x7f40500cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f40500cfc40 0x7f40500b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40500720d0 con 0x7f4050144dd0 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f4050072370 con 0x7f40500cfc40 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4057348640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4050072650 con 0x7f405006f870 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40550e4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4050144dd0 0x7f405006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50734/0 (socket says 172.21.15.22:50734) 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40550e4640 1 -- 172.21.15.22:0/2251133524 learned_addr learned my addr 172.21.15.22:0/2251133524 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 --2- 172.21.15.22:0/2251133524 >> v2:172.21.15.22:3300/0 conn(0x7f40500cfc40 0x7f40500b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40548e3640 1 --2- 172.21.15.22:0/2251133524 >> v2:172.21.15.183:3300/0 conn(0x7f405006f870 0x7f40500cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4047fff640 1 -- 172.21.15.22:0/2251133524 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411907278 0 0) 0x7f40500720d0 con 0x7f4050144dd0 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f4047fff640 1 -- 172.21.15.22:0/2251133524 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f403c003530 con 0x7f4050144dd0 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 -- 172.21.15.22:0/2251133524 >> v2:172.21.15.183:3300/0 conn(0x7f405006f870 msgr2=0x7f40500cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 --2- 172.21.15.22:0/2251133524 >> v2:172.21.15.183:3300/0 conn(0x7f405006f870 0x7f40500cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 -- 172.21.15.22:0/2251133524 >> v1:172.21.15.22:6789/0 conn(0x7f4050144dd0 legacy=0x7f405006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 -- 172.21.15.22:0/2251133524 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4050072930 con 0x7f40500cfc40 2024-04-27T07:54:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.093+0000 7f40558e5640 1 --2- 172.21.15.22:0/2251133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40500cfc40 0x7f40500b3dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f40500703f0 tx=0x7f404801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5181472f36ccfac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4047fff640 1 -- 172.21.15.22:0/2251133524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40480b2070 con 0x7f40500cfc40 2024-04-27T07:54:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4047fff640 1 -- 172.21.15.22:0/2251133524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40480a58c0 con 0x7f40500cfc40 2024-04-27T07:54:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4047fff640 1 -- 172.21.15.22:0/2251133524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40480a5c90 con 0x7f40500cfc40 2024-04-27T07:54:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/2251133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40500cfc40 msgr2=0x7f40500b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- 172.21.15.22:0/2251133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40500cfc40 0x7f40500b3dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f40500703f0 tx=0x7f404801c930 comp rx=0 tx=0).stop 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/2251133524 shutdown_connections 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- 172.21.15.22:0/2251133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40500cfc40 0x7f40500b3dd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- 172.21.15.22:0/2251133524 >> v2:172.21.15.183:3300/0 conn(0x7f405006f870 0x7f40500cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/2251133524 >> 172.21.15.22:0/2251133524 conn(0x7f4050067f10 msgr2=0x7f405006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/2251133524 shutdown_connections 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/2251133524 wait complete. 2024-04-27T07:54:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 Processor -- start 2024-04-27T07:54:08.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- start start 2024-04-27T07:54:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40500cfc40 0x7f4050140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 0x7f4050141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b4ca0 con 0x7f405006f870 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b4b20 con 0x7f4050144dd0 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f40500b4e20 con 0x7f40500cfc40 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57974/0 (socket says 172.21.15.22:57974) 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 -- 172.21.15.22:0/370287678 learned_addr learned my addr 172.21.15.22:0/370287678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40558e5640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 0x7f4050141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40548e3640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40500cfc40 0x7f4050140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 -- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40500cfc40 msgr2=0x7f4050140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40500cfc40 0x7f4050140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 -- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 msgr2=0x7f4050141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 0x7f4050141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 -- 172.21.15.22:0/370287678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405015ade0 con 0x7f405006f870 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40558e5640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 0x7f4050141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f40550e4640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f404000c3c0 tx=0x7f404000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040017070 con 0x7f405006f870 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4040005cf0 con 0x7f405006f870 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40500aa4a0 con 0x7f405006f870 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40400072f0 con 0x7f405006f870 2024-04-27T07:54:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.097+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40500a99c0 con 0x7f405006f870 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4050068c30 con 0x7f405006f870 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f404001a070 con 0x7f405006f870 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f4045ffb640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 0x7f403406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4040085ed0 con 0x7f405006f870 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f40548e3640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 0x7f403406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.101+0000 7f40548e3640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 0x7f403406a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f40500aa7f0 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.105+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4040021020 con 0x7f405006f870 2024-04-27T07:54:08.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.257+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f405015e8f0 con 0x7f4034068380 2024-04-27T07:54:08.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.257+0000 7f4045ffb640 1 -- 172.21.15.22:0/370287678 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13350 (secure 0 0 0) 0x7f405015e8f0 con 0x7f4034068380 2024-04-27T07:54:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:08.262 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:54:08.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 msgr2=0x7f403406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 0x7f403406a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f40500aa7f0 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-04-27T07:54:08.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 msgr2=0x7f405015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:08.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f404000c3c0 tx=0x7f404000c880 comp rx=0 tx=0).stop 2024-04-27T07:54:08.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 shutdown_connections 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7f4034068380 0x7f403406a840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050144dd0 0x7f4050141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40500cfc40 0x7f4050140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 --2- 172.21.15.22:0/370287678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006f870 0x7f405015a8a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 >> 172.21.15.22:0/370287678 conn(0x7f4050067f10 msgr2=0x7f405015da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 shutdown_connections 2024-04-27T07:54:08.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:08.261+0000 7f4057348640 1 -- 172.21.15.22:0/370287678 wait complete. 2024-04-27T07:54:08.279 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-04-27T07:54:08.218475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241468,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733252,"statfs":{"total":773094113280,"available":772846850048,"internally_reserved":0,"allocated":4595712,"data_stored":2520624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.194099"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-27T07:54:00.173893+0000","last_change":"2024-04-27T07:50:43.877777+0000","last_active":"2024-04-27T07:54:00.173893+0000","last_peered":"2024-04-27T07:54:00.173893+0000","last_clean":"2024-04-27T07:54:00.173893+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:54:00.173893+0000","last_undegraded":"2024-04-27T07:54:00.173893+0000","last_fullsized":"2024-04-27T07:54:00.173893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:50:42.666753+0000","last_clean_scrub_stamp":"2024-04-27T07:50:42.666753+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:31:33.579240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:54:08.283 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-27T07:50:42.666753+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=7, tm_min=50, tm_sec=42, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=7, tm_min=54, tm_sec=3, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T07:54:08.283 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T07:54:27.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:27.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:28.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:54:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 Processor -- start 2024-04-27T07:54:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 -- start start 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6600d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600cff70 con 0x7fc6600b7930 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600b3d30 con 0x7fc66015ebe0 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc665953640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc6600d4540 con 0x7fc6600d4790 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc6600b7930 0x7fc6600d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49646/0 (socket says 172.21.15.22:49646) 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65f7fe640 1 -- 172.21.15.22:0/2110797696 learned_addr learned my addr 172.21.15.22:0/2110797696 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65effd640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6600d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319405346 0 0) 0x7fc6600cff70 con 0x7fc6600b7930 2024-04-27T07:54:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc648003530 con 0x7fc6600b7930 2024-04-27T07:54:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2284923602 0 0) 0x7fc648003530 con 0x7fc6600b7930 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 msgr2=0x7fc6600d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6600d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 >> v1:172.21.15.183:6789/0 conn(0x7fc66015ebe0 legacy=0x7fc6600cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6600ba7a0 con 0x7fc6600b7930 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fc658001a20 con 0x7fc6600b7930 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 >> v1:172.21.15.22:6789/0 conn(0x7fc6600b7930 legacy=0x7fc6600d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc648003f30 0x7fc648004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648004d50 0x7fc6480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600d4540 con 0x7fc648003f30 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800bb10 con 0x7fc648004d50 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc64800bd10 con 0x7fc6600d4790 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc648003f30 0x7fc648004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65f7fe640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648004d50 0x7fc6480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.416+0000 7fc65effd640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 -- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 msgr2=0x7fc6480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6600d4790 0x7fc6480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 -- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648004d50 msgr2=0x7fc6480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648004d50 0x7fc6480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 -- 172.21.15.22:0/2110797696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc648003530 con 0x7fc648003f30 2024-04-27T07:54:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65f7fe640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648004d50 0x7fc6480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc648003f30 0x7fc648004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc654001a30 tx=0x7fc65401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99c0400dd286675 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540b5070 con 0x7fc648003f30 2024-04-27T07:54:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6540a8890 con 0x7fc648003f30 2024-04-27T07:54:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65dffb640 1 -- 172.21.15.22:0/2110797696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654001630 con 0x7fc648003f30 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 -- 172.21.15.22:0/2110797696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc648003f30 msgr2=0x7fc648004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 --2- 172.21.15.22:0/2110797696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc648003f30 0x7fc648004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc654001a30 tx=0x7fc65401ec70 comp rx=0 tx=0).stop 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc65f7fe640 1 -- 172.21.15.22:0/2110797696 reap_dead start 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 -- 172.21.15.22:0/2110797696 shutdown_connections 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 -- 172.21.15.22:0/2110797696 >> 172.21.15.22:0/2110797696 conn(0x7fc660067d10 msgr2=0x7fc660068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 -- 172.21.15.22:0/2110797696 shutdown_connections 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.420+0000 7fc665953640 1 -- 172.21.15.22:0/2110797696 wait complete. 2024-04-27T07:54:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 Processor -- start 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- start start 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 0x7fc66015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc660147470 0x7fc6601478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600d4260 con 0x7fc660155310 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600d3f80 con 0x7fc648003f30 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc6600ba120 con 0x7fc660147470 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50780/0 (socket says 172.21.15.22:50780) 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 -- 172.21.15.22:0/3427683027 learned_addr learned my addr 172.21.15.22:0/3427683027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65f7fe640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc660147470 0x7fc6601478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65effd640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 0x7fc66015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 -- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc660147470 msgr2=0x7fc6601478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc660147470 0x7fc6601478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 -- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 msgr2=0x7fc66015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 0x7fc66015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 -- 172.21.15.22:0/3427683027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc660148790 con 0x7fc660155310 2024-04-27T07:54:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65effd640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 0x7fc66015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc65e7fc640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc6540b3040 tx=0x7fc648005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540be020 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc654005070 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6601481d0 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540b5040 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc660160130 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc660068c30 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.424+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc6540c1020 con 0x7fc660155310 2024-04-27T07:54:28.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.428+0000 7fc647fff640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 0x7fc64c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.428+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc654123a60 con 0x7fc660155310 2024-04-27T07:54:28.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.428+0000 7fc65effd640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 0x7fc64c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.428+0000 7fc65effd640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 0x7fc64c06a840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc650009a20 tx=0x7fc650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.432+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6540c8030 con 0x7fc660155310 2024-04-27T07:54:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.588+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 --> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc660160510 con 0x7fc64c068380 2024-04-27T07:54:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.592+0000 7fc647fff640 1 -- 172.21.15.22:0/3427683027 <== mgr.4100 v2:172.21.15.183:6832/832018128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13357 (secure 0 0 0) 0x7fc660160510 con 0x7fc64c068380 2024-04-27T07:54:28.597 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-27T07:54:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 msgr2=0x7fc64c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 0x7fc64c06a840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc650009a20 tx=0x7fc650008040 comp rx=0 tx=0).stop 2024-04-27T07:54:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 msgr2=0x7fc660155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc6540b3040 tx=0x7fc648005840 comp rx=0 tx=0).stop 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 shutdown_connections 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fc64c068380 0x7fc64c06a840 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc660147470 0x7fc6601478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660155310 0x7fc660155790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 --2- 172.21.15.22:0/3427683027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648003f30 0x7fc66015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 >> 172.21.15.22:0/3427683027 conn(0x7fc660067d10 msgr2=0x7fc6600b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 shutdown_connections 2024-04-27T07:54:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.596+0000 7fc665953640 1 -- 172.21.15.22:0/3427683027 wait complete. 2024-04-27T07:54:28.615 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-27T07:54:28.223604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241456,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733264,"statfs":{"total":773094113280,"available":772846862336,"internally_reserved":0,"allocated":4595712,"data_stored":2520624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003050"},"pg_stats":[{"pgid":"1.0","version":"37'119","reported_seq":211,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-27T07:54:08.871551+0000","last_change":"2024-04-27T07:54:08.871722+0000","last_active":"2024-04-27T07:54:08.871551+0000","last_peered":"2024-04-27T07:54:08.871551+0000","last_clean":"2024-04-27T07:54:08.871551+0000","last_became_active":"2024-04-27T07:50:43.874870+0000","last_became_peered":"2024-04-27T07:50:43.874870+0000","last_unstale":"2024-04-27T07:54:08.871551+0000","last_undegraded":"2024-04-27T07:54:08.871551+0000","last_fullsized":"2024-04-27T07:54:08.871551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'119","last_scrub_stamp":"2024-04-27T07:54:08.871329+0000","last_deep_scrub":"37'119","last_deep_scrub_stamp":"2024-04-27T07:54:08.871329+0000","last_clean_scrub_stamp":"2024-04-27T07:54:08.871329+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:41:27.051537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":167486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1019904,"data_stored":757854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:54:28.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-27T07:54:28.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 Processor -- start 2024-04-27T07:54:28.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 -- start start 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fce840ce830 0x7fce8409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce8415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce840d0cf0 con 0x7fce840d0760 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fce840d0fd0 con 0x7fce840ce830 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce8a06f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce840d12b0 con 0x7fce8406de00 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce82ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fce840d0760 0x7fce8406d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49660/0 (socket says 172.21.15.22:49660) 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce82ffd640 1 -- 172.21.15.22:0/3771262916 learned_addr learned my addr 172.21.15.22:0/3771262916 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-04-27T07:54:28.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> v2:172.21.15.22:3300/0 conn(0x7fce840ce830 0x7fce8409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce83fff640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce8415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452016455 0 0) 0x7fce840d0cf0 con 0x7fce840d0760 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce70003530 con 0x7fce840d0760 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1426400635 0 0) 0x7fce70003530 con 0x7fce840d0760 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 msgr2=0x7fce8415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce8415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 >> v2:172.21.15.22:3300/0 conn(0x7fce840ce830 msgr2=0x7fce8409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 --2- 172.21.15.22:0/3771262916 >> v2:172.21.15.22:3300/0 conn(0x7fce840ce830 0x7fce8409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce840d1590 con 0x7fce840d0760 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4056722328 0 0) 0x7fce7c001680 con 0x7fce840d0760 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> v2:172.21.15.22:3300/0 conn(0x7fce840ce830 0x7fce8409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 >> v1:172.21.15.22:6789/0 conn(0x7fce840d0760 legacy=0x7fce8406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce70004d50 0x7fce700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce840d0fd0 con 0x7fce700039c0 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce840d12b0 con 0x7fce70004d50 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce7000b9d0 con 0x7fce8406de00 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.728+0000 7fce82ffd640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce83fff640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce70004d50 0x7fce700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 -- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 msgr2=0x7fce70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce8406de00 0x7fce70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 -- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce70004d50 msgr2=0x7fce700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce70004d50 0x7fce700051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 -- 172.21.15.22:0/3771262916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70003530 con 0x7fce700039c0 2024-04-27T07:54:28.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce837fe640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce70003de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e350 tx=0x7fce6c01ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c992cc946327e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0b2070 con 0x7fce700039c0 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c0ad440 con 0x7fce700039c0 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce827fc640 1 -- 172.21.15.22:0/3771262916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009de0 con 0x7fce700039c0 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 -- 172.21.15.22:0/3771262916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 msgr2=0x7fce70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 --2- 172.21.15.22:0/3771262916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce70003de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e350 tx=0x7fce6c01ce50 comp rx=0 tx=0).stop 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce83fff640 1 -- 172.21.15.22:0/3771262916 reap_dead start 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 -- 172.21.15.22:0/3771262916 shutdown_connections 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 -- 172.21.15.22:0/3771262916 >> 172.21.15.22:0/3771262916 conn(0x7fce840b4050 msgr2=0x7fce840b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 -- 172.21.15.22:0/3771262916 shutdown_connections 2024-04-27T07:54:28.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.732+0000 7fce8a06f640 1 -- 172.21.15.22:0/3771262916 wait complete. 2024-04-27T07:54:28.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 Processor -- start 2024-04-27T07:54:28.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- start start 2024-04-27T07:54:28.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 0x7fce84145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce84143470 0x7fce840cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce8415eb60 con 0x7fce700039c0 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce8415e9e0 con 0x7fce84157680 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fce8415ece0 con 0x7fce84143470 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50802/0 (socket says 172.21.15.22:50802) 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 -- 172.21.15.22:0/317582808 learned_addr learned my addr 172.21.15.22:0/317582808 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce82ffd640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 0x7fce84145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce83fff640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce84143470 0x7fce840cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 -- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce84143470 msgr2=0x7fce840cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce84143470 0x7fce840cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 -- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 msgr2=0x7fce84145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 0x7fce84145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 -- 172.21.15.22:0/317582808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce840cbfc0 con 0x7fce700039c0 2024-04-27T07:54:28.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce82ffd640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 0x7fce84145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce837fe640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fce6c01cc80 tx=0x7fce6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0b2040 con 0x7fce700039c0 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c0b6430 con 0x7fce700039c0 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce840cc2a0 con 0x7fce700039c0 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009de0 con 0x7fce700039c0 2024-04-27T07:54:28.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce84146400 con 0x7fce700039c0 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.736+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8404ff10 con 0x7fce700039c0 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.740+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce6c0bd020 con 0x7fce700039c0 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.740+0000 7fce7bfff640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 0x7fce6406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.740+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fce6c121300 con 0x7fce700039c0 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.740+0000 7fce82ffd640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 0x7fce6406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:28.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.740+0000 7fce82ffd640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 0x7fce6406a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce8415dee0 tx=0x7fce7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:28.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.744+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce6c0c4020 con 0x7fce700039c0 2024-04-27T07:54:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.904+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fce84158e40 con 0x7fce700039c0 2024-04-27T07:54:28.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.912+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fce6c016b90 con 0x7fce700039c0 2024-04-27T07:54:28.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.912+0000 7fce7bfff640 1 -- 172.21.15.22:0/317582808 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce6c0f1bc0 con 0x7fce700039c0 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 msgr2=0x7fce6406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 0x7fce6406a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce8415dee0 tx=0x7fce7c009040 comp rx=0 tx=0).stop 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 msgr2=0x7fce84156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fce6c01cc80 tx=0x7fce6c009750 comp rx=0 tx=0).stop 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 shutdown_connections 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fce640683b0 0x7fce6406a870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce84143470 0x7fce840cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce84157680 0x7fce84145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 --2- 172.21.15.22:0/317582808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce700039c0 0x7fce84156f60 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fce6c01cc80 tx=0x7fce6c009750 comp rx=0 tx=0).stop 2024-04-27T07:54:28.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 >> 172.21.15.22:0/317582808 conn(0x7fce840b4050 msgr2=0x7fce8406d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:28.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 shutdown_connections 2024-04-27T07:54:28.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:28.916+0000 7fce8a06f640 1 -- 172.21.15.22:0/317582808 wait complete. 2024-04-27T07:54:28.936 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:54:28.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:54:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 Processor -- start 2024-04-27T07:54:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 -- start start 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb44415ca90 0x7fb44409b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb4440cef10 0x7fb4440cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4440cf8e0 con 0x7fb44415c6b0 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb444072300 con 0x7fb44415ca90 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb44bc76640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440b1e70 con 0x7fb4440cef10 2024-04-27T07:54:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb44415ca90 0x7fb44409b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb44415ca90 0x7fb44409b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50818/0 (socket says 172.21.15.22:50818) 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 -- 172.21.15.22:0/3910030463 learned_addr learned my addr 172.21.15.22:0/3910030463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 -- 172.21.15.22:0/3910030463 >> v2:172.21.15.183:3300/0 conn(0x7fb4440cef10 msgr2=0x7fb4440cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 --2- 172.21.15.22:0/3910030463 >> v2:172.21.15.183:3300/0 conn(0x7fb4440cef10 0x7fb4440cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 -- 172.21.15.22:0/3910030463 >> v1:172.21.15.183:6789/0 conn(0x7fb44415c6b0 legacy=0x7fb444070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:54:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 -- 172.21.15.22:0/3910030463 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb444071260 con 0x7fb44415ca90 2024-04-27T07:54:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.044+0000 7fb449211640 1 --2- 172.21.15.22:0/3910030463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44415ca90 0x7fb44409b5d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb4440ccbb0 tx=0x7fb43401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753c482bbc5d2138 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb448a10640 1 -- 172.21.15.22:0/3910030463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4340b5070 con 0x7fb44415ca90 2024-04-27T07:54:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb448a10640 1 -- 172.21.15.22:0/3910030463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb4340a89d0 con 0x7fb44415ca90 2024-04-27T07:54:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb448a10640 1 -- 172.21.15.22:0/3910030463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4340a8dc0 con 0x7fb44415ca90 2024-04-27T07:54:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- 172.21.15.22:0/3910030463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44415ca90 msgr2=0x7fb44409b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3910030463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44415ca90 0x7fb44409b5d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb4440ccbb0 tx=0x7fb43401ecd0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- 172.21.15.22:0/3910030463 shutdown_connections 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3910030463 >> v2:172.21.15.183:3300/0 conn(0x7fb4440cef10 0x7fb4440cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3910030463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44415ca90 0x7fb44409b5d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- 172.21.15.22:0/3910030463 >> 172.21.15.22:0/3910030463 conn(0x7fb4440d25d0 msgr2=0x7fb4440d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- 172.21.15.22:0/3910030463 shutdown_connections 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- 172.21.15.22:0/3910030463 wait complete. 2024-04-27T07:54:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 Processor -- start 2024-04-27T07:54:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- start start 2024-04-27T07:54:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb44415c6b0 0x7fb4440ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 0x7fb4441580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb444071b00 con 0x7fb4440cef10 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb444071820 con 0x7fb44415c6b0 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44bc76640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb444071de0 con 0x7fb44415ca90 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb449a12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb449a12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50830/0 (socket says 172.21.15.22:50830) 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb449a12640 1 -- 172.21.15.22:0/3121217406 learned_addr learned my addr 172.21.15.22:0/3121217406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb449211640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb44415c6b0 0x7fb4440ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.048+0000 7fb44a213640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 0x7fb4441580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 -- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 msgr2=0x7fb4441580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 0x7fb4441580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 -- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb44415c6b0 msgr2=0x7fb4440ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb44415c6b0 0x7fb4440ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 -- 172.21.15.22:0/3121217406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4440cf6b0 con 0x7fb4440cef10 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb44a213640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 0x7fb4441580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449a12640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c7d0 tx=0x7fb44000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440017070 con 0x7fb4440cef10 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb440003db0 con 0x7fb4440cef10 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400052d0 con 0x7fb4440cef10 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4440ced10 con 0x7fb4440cef10 2024-04-27T07:54:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4440b2440 con 0x7fb4440cef10 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444069340 con 0x7fb4440cef10 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb440005470 con 0x7fb4440cef10 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 0x7fb42806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb440085c60 con 0x7fb4440cef10 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449211640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 0x7fb42806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:54:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.052+0000 7fb449211640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 0x7fb42806a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b2000 tx=0x7fb434008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:54:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.056+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb440056520 con 0x7fb4440cef10 2024-04-27T07:54:29.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.248+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4440c8e30 con 0x7fb4440cef10 2024-04-27T07:54:29.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.248+0000 7fb43a7fc640 1 -- 172.21.15.22:0/3121217406 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3875 (secure 0 0 0) 0x7fb440055e20 con 0x7fb4440cef10 2024-04-27T07:54:29.253 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-27T07:54:29.253 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1912139093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1912139093},{"type":"v1","addr":"172.21.15.183:6835","nonce":1912139093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2097582637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2097582637},{"type":"v1","addr":"172.21.15.22:6835","nonce":2097582637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1993559612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1993559612},{"type":"v1","addr":"172.21.15.183:6837","nonce":1993559612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3346541446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3346541446},{"type":"v1","addr":"172.21.15.22:6834","nonce":3346541446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-27T07:54:29.254 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 18 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 msgr2=0x7fb42806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 0x7fb42806a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b2000 tx=0x7fb434008040 comp rx=0 tx=0).stop 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 msgr2=0x7fb44414e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c7d0 tx=0x7fb44000cc90 comp rx=0 tx=0).stop 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 shutdown_connections 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:6832/832018128,v1:172.21.15.183:6833/832018128] conn(0x7fb428068380 0x7fb42806a840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb44415ca90 0x7fb4441580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb44415c6b0 0x7fb4440ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 --2- 172.21.15.22:0/3121217406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4440cef10 0x7fb44414e8d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:54:29.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 >> 172.21.15.22:0/3121217406 conn(0x7fb4440d25d0 msgr2=0x7fb4440ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:54:29.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 shutdown_connections 2024-04-27T07:54:29.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-27T07:54:29.252+0000 7fb44bc76640 1 -- 172.21.15.22:0/3121217406 wait complete. 2024-04-27T07:54:29.271 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-27T07:54:29.271 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-27T07:54:29.271 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-27T07:54:29.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:29.282 INFO:tasks.ceph.mds.a:Stopped 2024-04-27T07:54:29.282 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-27T07:54:29.283 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:29.347 INFO:tasks.ceph.mds.c:Stopped 2024-04-27T07:54:29.347 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-27T07:54:29.347 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:29.366 INFO:tasks.ceph.mds.b:Stopped 2024-04-27T07:54:29.366 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-27T07:54:29.366 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:29.431 INFO:tasks.ceph.mds.d:Stopped 2024-04-27T07:54:29.431 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-27T07:54:29.431 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-27T07:54:29.431 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:29.485 INFO:tasks.ceph.osd.0:Stopped 2024-04-27T07:54:29.485 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-27T07:54:29.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:33.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-27T07:54:35.586 INFO:tasks.ceph.osd.1:Stopped 2024-04-27T07:54:35.587 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-27T07:54:35.587 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:35.641 INFO:tasks.ceph.osd.2:Stopped 2024-04-27T07:54:35.642 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-27T07:54:35.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:35.738 INFO:tasks.ceph.osd.3:Stopped 2024-04-27T07:54:35.738 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-27T07:54:35.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:35.803 INFO:tasks.ceph.osd.4:Stopped 2024-04-27T07:54:35.803 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-27T07:54:35.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:35.889 INFO:tasks.ceph.osd.5:Stopped 2024-04-27T07:54:35.889 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-27T07:54:35.889 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:35.971 INFO:tasks.ceph.osd.6:Stopped 2024-04-27T07:54:35.971 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-27T07:54:35.971 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.062 INFO:tasks.ceph.osd.7:Stopped 2024-04-27T07:54:36.062 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-27T07:54:36.062 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-27T07:54:36.062 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.089 INFO:tasks.ceph.mgr.y:Stopped 2024-04-27T07:54:36.089 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-27T07:54:36.090 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.119 INFO:tasks.ceph.mgr.x:Stopped 2024-04-27T07:54:36.119 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-27T07:54:36.119 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-27T07:54:36.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.148 INFO:tasks.ceph.mon.a:Stopped 2024-04-27T07:54:36.148 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-27T07:54:36.148 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.172 INFO:tasks.ceph.mon.b:Stopped 2024-04-27T07:54:36.173 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-27T07:54:36.173 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:54:36.229 INFO:tasks.ceph.mon.c:Stopped 2024-04-27T07:54:36.241 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-27T07:54:36.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-04-27T07:54:36.259 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:54:36.260 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-27T07:54:36.623 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:54:36.623 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-27T07:54:36.994 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:54:36.994 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-27T07:54:37.177 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:54:37.178 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-27T07:54:37.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:54:37.353 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-27T07:54:37.605 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:54:37.605 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-27T07:54:37.814 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:54:37.814 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-27T07:54:38.011 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:54:38.011 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-27T07:54:38.191 INFO:tasks.ceph:Archiving mon data... 2024-04-27T07:54:38.193 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/data/mon.a.tgz 2024-04-27T07:54:38.193 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-04-27T07:54:38.199 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.X5csfA4fqe 2024-04-27T07:54:38.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.X5csfA4fqe 2024-04-27T07:54:38.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.X5csfA4fqe 2024-04-27T07:54:38.384 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.X5csfA4fqe is 305KB 2024-04-27T07:54:38.445 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.X5csfA4fqe 2024-04-27T07:54:38.451 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/data/mon.b.tgz 2024-04-27T07:54:38.451 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-04-27T07:54:38.457 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.4yoDxK23Iw 2024-04-27T07:54:38.457 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.4yoDxK23Iw 2024-04-27T07:54:38.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.4yoDxK23Iw 2024-04-27T07:54:38.614 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-27T07:54:38.660 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.4yoDxK23Iw is 409KB 2024-04-27T07:54:38.718 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.4yoDxK23Iw 2024-04-27T07:54:38.726 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/data/mon.c.tgz 2024-04-27T07:54:38.726 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-04-27T07:54:38.773 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.Z9TkNr3svv 2024-04-27T07:54:38.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Z9TkNr3svv 2024-04-27T07:54:38.910 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.Z9TkNr3svv 2024-04-27T07:54:38.973 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.Z9TkNr3svv is 409KB 2024-04-27T07:54:39.034 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.Z9TkNr3svv 2024-04-27T07:54:39.040 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-27T07:54:39.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T07:54:39.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T07:54:39.103 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T07:54:39.149 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T07:54:39.149 INFO:tasks.ceph:Archiving crash dumps... 2024-04-27T07:54:39.151 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi022/crash 2024-04-27T07:54:39.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T07:54:39.167 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi183/crash 2024-04-27T07:54:39.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T07:54:39.205 INFO:tasks.ceph:Shutting down logrotate 2024-04-27T07:54:39.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:39.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:39.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:39.310 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:39.322 INFO:tasks.ceph:Compressing logs... 2024-04-27T07:54:39.322 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T07:54:39.356 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T07:54:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32240.log 2024-04-27T07:54:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75358.log 2024-04-27T07:54:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-04-27T07:54:39.369 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-04-27T07:54:39.369 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32240.log.gz 2024-04-27T07:54:39.369 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75358.log: /var/log/ceph/ceph-client.admin.35457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-04-27T07:54:39.369 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-04-27T07:54:39.369 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34529.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75358.log.gz 2024-04-27T07:54:39.370 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35982.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33443.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35982.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75917.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35982.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76411.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33443.log: /var/log/ceph/ceph-client.admin.75917.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33443.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76333.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75917.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76411.log.gz 2024-04-27T07:54:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76333.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-27T07:54:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34477.log 2024-04-27T07:54:39.374 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76333.log.gz 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22642.log 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32742.log 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.tmp-client.admin.22642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75811.log 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22642.log.gz 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32742.log: /var/log/ceph/ceph-client.admin.32947.log 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75462.log 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32742.log.gz 2024-04-27T07:54:39.375 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75811.log.gz 2024-04-27T07:54:39.378 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32663.log 2024-04-27T07:54:39.378 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32947.log.gz 2024-04-27T07:54:39.378 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75462.log: /var/log/ceph/ceph-client.admin.32663.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75462.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75691.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32663.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32934.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76047.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75691.log: /var/log/ceph/ceph-client.admin.32934.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75691.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33651.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32934.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34243.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33599.log 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33651.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76047.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34243.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-27T07:54:39.379 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33651.log.gz 2024-04-27T07:54:39.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76361.log 2024-04-27T07:54:39.380 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34243.log.gz 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27705.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27729.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.27705.log: /var/log/ceph/ceph-client.admin.27729.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.27729.log.gz 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr: /var/log/ceph/ceph-client.1.33296.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27705.log.gz 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27801.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.33296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27753.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.27801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27849.log 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27801.log.gz 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.27753.log: 90.2% -- replaced with /var/log/ceph/ceph-client.1.33296.log.gz 2024-04-27T07:54:39.398 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27753.log.gz 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.27849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27777.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.5.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27849.log.gz 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27825.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.27777.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27777.log.gz/var/log/ceph/ceph-client.admin.27825.log: 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27825.log.gz/var/log/ceph/ceph-mgr.x.log: 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33415.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-27T07:54:39.399 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.3.33415.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.d.log: 90.7% -- replaced with /var/log/ceph/ceph-client.3.33415.log.gz 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.b.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-27T07:54:39.400 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27873.log 2024-04-27T07:54:39.410 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32731.log 2024-04-27T07:54:39.410 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-04-27T07:54:39.410 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33599.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34321.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76361.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32731.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32737.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36497.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34321.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32349.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32737.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35930.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.32737.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32136.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35930.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32349.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34685.log 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35930.log.gz 2024-04-27T07:54:39.411 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36341.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32136.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36060.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34685.log: /var/log/ceph/ceph-client.admin.36341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32322.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34685.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36341.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36060.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76255.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36034.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.32322.log.gz --verbose 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.32689.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76255.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32724.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76255.log.gz 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32689.log: /var/log/ceph/ceph-client.admin.32724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33703.log 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34981.log: gzip 88.2% -5 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32724.log.gz --verbose 2024-04-27T07:54:39.412 INFO:teuthology.orchestra.run.smithi022.stderr: -- -- replaced with /var/log/ceph/ceph-client.admin.32689.log.gz /var/log/ceph/ceph-client.admin.34955.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33703.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33703.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34955.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33547.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34746.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33547.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34746.log: /var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32479.log 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34746.log.gz 2024-04-27T07:54:39.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32479.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76099.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32479.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76177.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76177.log: -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76099.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.36776.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35405.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76177.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-04-27T07:54:39.414 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34107.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35592.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35405.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.44163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36086.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75969.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36086.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.34903.log -- replaced with /var/log/ceph/ceph-client.admin.34107.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75969.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36086.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34503.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75969.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-04-27T07:54:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44189.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35800.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34503.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33156.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.44189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33521.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32078.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33521.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33156.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33521.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34165.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32078.log.gz 2024-04-27T07:54:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75436.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31979.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34555.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75436.log: -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75384.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75436.log.gz 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34555.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31979.log.gz 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34217.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34555.log.gz 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:gzip 88.5% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.75384.log.gz /var/log/ceph/ceph-client.admin.35352.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34217.log: /var/log/ceph/ceph-client.admin.34451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-04-27T07:54:39.417 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34217.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35352.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35116.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35352.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76073.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33157.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35116.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76073.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34347.log: /var/log/ceph/ceph-client.admin.33157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35274.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36724.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33157.log.gz 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-04-27T07:54:39.418 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34581.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35431.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35274.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35774.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34581.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32743.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36750.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32743.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36960.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32743.log.gz 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32946.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36138.log 2024-04-27T07:54:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-04-27T07:54:39.420 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2%/var/log/ceph/ceph-client.admin.36138.log: -- replaced with /var/log/ceph/ceph-client.admin.32946.log.gz 2024-04-27T07:54:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32453.log 2024-04-27T07:54:39.420 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36138.log.gz 2024-04-27T07:54:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-27T07:54:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32453.log: /var/log/ceph/ceph-client.admin.33469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34633.log 2024-04-27T07:54:39.428 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27873.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27873.log.gz 2024-04-27T07:54:39.428 INFO:teuthology.orchestra.run.smithi183.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33339.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32453.log.gz 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76229.log 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32728.log 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34633.log.gz/var/log/ceph/ceph-client.admin.76229.log: 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32637.log 2024-04-27T07:54:39.438 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76229.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32728.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32728.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.0.36960.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32637.log: /var/log/ceph/ceph-client.admin.36672.log: gzip -5 89.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32637.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: /var/log/ceph/ceph-client.admin.36854.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35878.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36170.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33149.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36854.log: /var/log/ceph/ceph-client.admin.36170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75618.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%/var/log/ceph/ceph-client.admin.33149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33159.log 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36170.log.gz 2024-04-27T07:54:39.439 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33149.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75618.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75618.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75717.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33159.log: /var/log/ceph/ceph-client.admin.75717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%/var/log/ceph/ceph-client.admin.35540.log: -- replaced with /var/log/ceph/ceph-client.admin.75717.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33159.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz/var/log/ceph/ceph-client.admin.34850.log: 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32611.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.32611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32938.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36393.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-04-27T07:54:39.440 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32611.log.gz 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32938.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32938.log.gz 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33962.log 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27453.log 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33962.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33962.log.gz 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75514.log 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36471.log 2024-04-27T07:54:39.441 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75514.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.27453.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75514.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34269.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%/var/log/ceph/ceph-client.admin.36614.log: -- replaced with /var/log/ceph/ceph-client.admin.36471.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35246.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34607.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34269.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75644.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35246.log: /var/log/ceph/ceph-client.admin.34607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35509.log 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75644.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35246.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34607.log.gz 2024-04-27T07:54:39.442 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75891.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75644.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35509.log: /var/log/ceph/ceph-client.admin.75891.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35826.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35509.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75891.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36553.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36112.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34295.log: -- replaced with /var/log/ceph/ceph-client.admin.36553.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27672.log 2024-04-27T07:54:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32375.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36112.log.gz/var/log/ceph/ceph-client.admin.34659.log: 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27672.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27672.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33677.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35326.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33677.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32375.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33677.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75769.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33154.log: /var/log/ceph/ceph-client.admin.75769.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35326.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.75769.log.gz --verbose 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.35038.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-04-27T07:54:39.444 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2%/var/log/ceph/ceph-client.admin.35038.log: -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76307.log 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35038.log.gz 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32162.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32188.log 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33729.log 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76307.log: /var/log/ceph/ceph-client.admin.32188.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.76307.log.gz /var/log/ceph/ceph-client.admin.32505.log 2024-04-27T07:54:39.445 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.453 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32188.log.gz 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33729.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.33729.log.gz 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33391.log 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%/var/log/ceph/ceph-client.admin.33391.log: -- replaced with /var/log/ceph/ceph-client.admin.32505.log.gz 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76151.log 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33391.log.gz 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35090.log 2024-04-27T07:54:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35483.log 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz/var/log/ceph/ceph-client.admin.76151.log: 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35090.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76151.log.gz 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35090.log.gz 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35194.log 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33365.log 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7%/var/log/ceph/ceph-client.admin.35194.log: -- replaced with /var/log/ceph/ceph-client.admin.35483.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32583.log 2024-04-27T07:54:39.455 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35194.log.gz 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36640.log 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75865.log 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33365.log.gz/var/log/ceph/ceph-client.admin.32583.log: 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36640.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32583.log.gz 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36640.log.gz 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35748.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75865.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32933.log 2024-04-27T07:54:39.456 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35064.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32933.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32933.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33625.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34798.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33625.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35064.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32427.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3%/var/log/ceph/ceph-client.admin.34798.log: -- replaced with /var/log/ceph/ceph-client.admin.33625.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32427.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34798.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32427.log.gz 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-04-27T07:54:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-27T07:54:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.35956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-04-27T07:54:39.458 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-04-27T07:54:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.c.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35012.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36802.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35012.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35012.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33784.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33912.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33784.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33784.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33131.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.33912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33131.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33131.log.gz 2024-04-27T07:54:39.465 INFO:teuthology.orchestra.run.smithi022.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.33912.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32012.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32531.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32012.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32012.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35852.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33151.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35852.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32531.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35852.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35142.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33151.log.gz 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75488.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32401.log 2024-04-27T07:54:39.466 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7%/var/log/ceph/ceph-client.admin.75488.log: -- replaced with /var/log/ceph/ceph-client.admin.35142.log.gz 2024-04-27T07:54:39.467 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75488.log.gz 2024-04-27T07:54:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75592.log 2024-04-27T07:54:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36588.log 2024-04-27T07:54:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32401.log: /var/log/ceph/ceph-client.admin.75592.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32401.log.gz 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75592.log.gz 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36588.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36588.log.gz 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34772.log 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36419.log: /var/log/ceph/ceph-client.admin.36008.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75839.log 2024-04-27T07:54:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34772.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34772.log.gz 2024-04-27T07:54:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32939.log 2024-04-27T07:54:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36288.log 2024-04-27T07:54:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75839.log: /var/log/ceph/ceph-client.admin.32939.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75839.log.gz 2024-04-27T07:54:39.471 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32939.log.gz 2024-04-27T07:54:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35220.log 2024-04-27T07:54:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36288.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36288.log.gz 2024-04-27T07:54:39.472 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75743.log 2024-04-27T07:54:39.472 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33573.log 2024-04-27T07:54:39.472 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35220.log: /var/log/ceph/ceph-client.admin.75743.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35220.log.gz 2024-04-27T07:54:39.473 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75743.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35168.log 2024-04-27T07:54:39.473 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.473 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33573.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33573.log.gz 2024-04-27T07:54:39.474 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75540.log 2024-04-27T07:54:39.474 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-04-27T07:54:39.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35168.log: /var/log/ceph/ceph-client.admin.75540.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35168.log.gz 2024-04-27T07:54:39.475 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75540.log.gz 2024-04-27T07:54:39.475 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36236.log 2024-04-27T07:54:39.476 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34929.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-04-27T07:54:39.476 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32951.log 2024-04-27T07:54:39.477 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32740.log 2024-04-27T07:54:39.477 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36236.log: /var/log/ceph/ceph-client.admin.32951.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36236.log.gz 2024-04-27T07:54:39.478 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32951.log.gz 2024-04-27T07:54:39.478 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-04-27T07:54:39.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32740.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76203.log 2024-04-27T07:54:39.479 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32740.log.gz 2024-04-27T07:54:39.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-04-27T07:54:39.480 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76203.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-04-27T07:54:39.480 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76203.log.gz 2024-04-27T07:54:39.480 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32296.log 2024-04-27T07:54:39.481 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75943.log 2024-04-27T07:54:39.481 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36445.log: /var/log/ceph/ceph-client.admin.32296.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-04-27T07:54:39.481 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32296.log.gz 2024-04-27T07:54:39.482 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75995.log 2024-04-27T07:54:39.482 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76385.log 2024-04-27T07:54:39.482 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75943.log: /var/log/ceph/ceph-client.admin.75995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75943.log.gz 2024-04-27T07:54:39.483 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32730.log 2024-04-27T07:54:39.483 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75995.log.gz 2024-04-27T07:54:39.483 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76385.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76385.log.gz 2024-04-27T07:54:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32268.log 2024-04-27T07:54:39.485 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75410.log 2024-04-27T07:54:39.485 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0%/var/log/ceph/ceph-client.admin.32268.log: -- replaced with /var/log/ceph/ceph-client.admin.32730.log.gz 2024-04-27T07:54:39.485 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32268.log.gz 2024-04-27T07:54:39.485 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32214.log 2024-04-27T07:54:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33146.log 2024-04-27T07:54:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75410.log: /var/log/ceph/ceph-client.admin.32214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75410.log.gz 2024-04-27T07:54:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32557.log 2024-04-27T07:54:39.487 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32214.log.gz 2024-04-27T07:54:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33146.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33146.log.gz 2024-04-27T07:54:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76125.log 2024-04-27T07:54:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-04-27T07:54:39.489 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32557.log: /var/log/ceph/ceph-client.admin.76125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32557.log.gz 2024-04-27T07:54:39.489 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76125.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75566.log 2024-04-27T07:54:39.489 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.490 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36828.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-04-27T07:54:39.490 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76281.log 2024-04-27T07:54:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 2024-04-27T07:54:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.75566.log: /var/log/ceph/ceph-client.admin.76281.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75566.log.gz 2024-04-27T07:54:39.492 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76281.log.gz 2024-04-27T07:54:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-04-27T07:54:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35696.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz 2024-04-27T07:54:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36262.log 2024-04-27T07:54:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76021.log 2024-04-27T07:54:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35618.log: /var/log/ceph/ceph-client.admin.36262.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-04-27T07:54:39.494 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36525.log 2024-04-27T07:54:39.494 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-04-27T07:54:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.76021.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76021.log.gz 2024-04-27T07:54:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-04-27T07:54:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27562.log 2024-04-27T07:54:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36525.log: /var/log/ceph/ceph-client.admin.35904.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36525.log.gz 2024-04-27T07:54:39.496 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34824.log 2024-04-27T07:54:39.497 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27562.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27562.log.gz 2024-04-27T07:54:39.498 INFO:teuthology.orchestra.run.smithi022.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-27T07:54:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34057.log 2024-04-27T07:54:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32948.log 2024-04-27T07:54:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34824.log: /var/log/ceph/ceph-client.2.34057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35300.log 2024-04-27T07:54:39.499 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1%/var/log/ceph/ceph-client.admin.32948.log: -- replaced with /var/log/ceph/ceph-client.admin.34824.log.gz 2024-04-27T07:54:39.499 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32948.log.gz 2024-04-27T07:54:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35300.log: 91.5% -- replaced with /var/log/ceph/ceph-client.2.34057.log.gz 2024-04-27T07:54:39.500 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35300.log.gz 2024-04-27T07:54:39.508 INFO:teuthology.orchestra.run.smithi183.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-27T07:54:39.508 INFO:teuthology.orchestra.run.smithi183.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-27T07:54:39.518 INFO:teuthology.orchestra.run.smithi022.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-27T07:54:39.537 INFO:teuthology.orchestra.run.smithi022.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-27T07:54:39.543 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-27T07:54:39.557 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-27T07:54:39.565 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-27T07:54:39.571 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-27T07:54:39.572 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-27T07:54:39.572 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m0.204s 2024-04-27T07:54:39.573 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m0.994s 2024-04-27T07:54:39.573 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.056s 2024-04-27T07:54:39.573 INFO:teuthology.orchestra.run.smithi022.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-27T07:54:39.588 INFO:teuthology.orchestra.run.smithi022.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-27T07:54:39.661 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-27T07:54:39.662 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-27T07:54:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m0.303s 2024-04-27T07:54:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m1.120s 2024-04-27T07:54:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.148s 2024-04-27T07:54:39.662 INFO:tasks.ceph:Archiving logs... 2024-04-27T07:54:39.662 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi022/log 2024-04-27T07:54:39.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T07:54:40.000 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi183/log 2024-04-27T07:54:40.001 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T07:54:40.129 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T07:54:40.154 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T07:54:40.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T07:54:40.156 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T07:54:40.223 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T07:54:40.232 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T07:54:40.232 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:54:40.235 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 177 0.091 -1.952 1.032 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout:+hv02.front.sepi 63.231.80.2 3 u 49 64 177 0.100 +0.021 0.601 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout:*hv03.front.sepi 74.6.168.72 3 u 51 64 177 0.066 -0.834 0.649 2024-04-27T07:54:40.245 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:54:40.247 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:54:40.247 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-04-27T07:54:40.247 INFO:teuthology.orchestra.run.smithi183.stdout:+hv01.front.sepi 67.205.162.81 3 u 46 64 177 0.142 -2.057 0.926 2024-04-27T07:54:40.247 INFO:teuthology.orchestra.run.smithi183.stdout:+hv02.front.sepi 63.231.80.2 3 u 54 64 177 0.101 -0.698 0.759 2024-04-27T07:54:40.247 INFO:teuthology.orchestra.run.smithi183.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 177 0.086 -1.509 0.797 2024-04-27T07:54:40.248 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:54:40.248 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T07:54:40.258 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T07:54:40.258 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T07:54:40.282 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T07:54:40.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T07:54:40.324 INFO:teuthology.task.internal:Duration was 882.409447 seconds 2024-04-27T07:54:40.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T07:54:40.334 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T07:54:40.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T07:54:40.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T07:54:40.374 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T07:54:40.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-04-27T07:54:40.374 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T07:54:40.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-04-27T07:54:40.424 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T07:54:40.434 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T07:54:40.435 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T07:54:40.468 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T07:54:40.502 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T07:54:40.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T07:54:40.540 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T07:54:40.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T07:54:40.612 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T07:54:40.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T07:54:40.626 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T07:54:40.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T07:54:40.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:54:40.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:54:40.690 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-04-27T07:54:40.693 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-04-27T07:54:40.706 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:54:40.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:54:40.751 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:54:40.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:54:40.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T07:54:40.770 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T07:54:40.771 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi022 2024-04-27T07:54:40.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T07:54:40.815 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096/remote/smithi183 2024-04-27T07:54:40.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T07:54:40.837 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T07:54:40.837 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T07:54:40.860 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T07:54:40.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T07:54:40.893 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T07:54:40.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T07:54:40.907 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T07:54:40.907 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T07:54:40.910 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T07:54:40.913 INFO:teuthology.orchestra.run.smithi022.stdout: 658639 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 27 07:54 /home/ubuntu/cephtest 2024-04-27T07:54:40.913 INFO:teuthology.orchestra.run.smithi022.stdout: 796522 4 d--------- 2 ubuntu ubuntu 4096 Apr 27 07:52 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:54:40.913 INFO:teuthology.orchestra.run.smithi022.stderr:find: '/home/ubuntu/cephtest/mnt.0': Permission denied 2024-04-27T07:54:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-27T07:54:40.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:54:40.926 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi022 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-27T07:54:40.927 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T07:54:40.937 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T07:54:40.965 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi022 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-27T07:54:40.965 INFO:teuthology.nuke:Checking targets against current locks 2024-04-27T07:54:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 07:54 /home/ubuntu/cephtest 2024-04-27T07:54:40.989 DEBUG:teuthology.nuke:shortname: smithi022 2024-04-27T07:54:40.990 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T07:54:41.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 07:31:40.686952', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBK5//N9I5E8X52d9w0riAGxZo/yEWmWVfz5IcTd0tHDDmpIi0mfkrfZpCBvNn7ogMpSAenSCGWIcXBnCuYlgvc='} 2024-04-27T07:54:41.019 DEBUG:teuthology.nuke:shortname: smithi183 2024-04-27T07:54:41.019 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T07:54:41.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672096', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 07:31:40.688379', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII9zDJkhPI0/3gDcjcctfIsRaLoXjYWnEsO4xBm3NFpKrd5ss4yUnjoi6O8bJpJ1kOqbgWwtLoG3PIc9Eesj4k='} 2024-04-27T07:54:41.064 INFO:teuthology.orchestra.console.smithi022:Power off 2024-04-27T07:54:41.065 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T07:54:41.087 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-04-27T07:54:41.087 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:41.103 INFO:teuthology.orchestra.console.smithi183:Power off 2024-04-27T07:54:41.103 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T07:54:41.112 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-27T07:54:41.126 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-04-27T07:54:41.126 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:41.147 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-27T07:54:45.114 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:45.148 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:45.224 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-27T07:54:45.259 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-27T07:54:49.225 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:49.260 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:54:49.335 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-04-27T07:54:49.370 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-04-27T07:54:49.436 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-04-27T07:54:49.471 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-04-27T07:54:49.575 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 882.4094474315643 failure_reason: 'Command failed on smithi022 with status 1: ''find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest''' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-27T07:54:49.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T07:54:49.665 INFO:teuthology.run:FAIL